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Executive Summary 



Benchmarking is a form of assessment that establishes external standards to which internal 
processes may be compared. The New York State School Report Card provides a vehicle for 
benchmarking with respect to student achievement. In this study, we have developed additional 
tools for making external comparisons with respect to achievement and have added tools for 
assessing fiscal policy and efficiency, i.e., comparing outcomes (achievement) with the amount of 
resources consumed in the schooling process. 

While this study used data available for the school years 1993-94 through 1995-96, the 
procedures and reporting devices can be adapted and updated as new information becomes 
available. Achievement data were obtained from the 1997 New York State School Report Card 
data base. Financial data for the 1995-96 school year were derived from Financing Our Schools- 
1997, prepared by the Erie County No. One Board of Cooperative Educational Services. The 
cases were limited to the 101 school districts and their schools in the western region of the state 
containing the two county Buffalo metropolitan area and six rural counties, three of whom 
contain small cities. The region is geographically about the size of the state of Connecticut and 
contains approximately 1.5 million people. 

Types of Measures Developed 

Two types of outcome measures were developed: average student achievement and average 
school and district effect. Average student achievement is measured by four factor scores at the 
district level and six average scores at the school level. Both sets of scores combine results of 
state examinations in reading, writing, mathematics, science and social studies given in grades 3, 

4, 5, 6 and 8 and for fifteen Regents examinations given in high school. School district effect is 
estimated through a multiple regression procedure that statistically removes co-variation of family 
and environmental characteristics from the student achievement scores. Input measures consist of 
expenditures (adjusted for within region variations in teacher costs) per pupil units (adjusted for 
variation in pupil characteristics) and pupil staff ratios. School district efficiency is measured 
using a linear programming technique known as data envelopment analysis (DEA), a modified 
quadriform approach, and a ratio approach. School efficiency is measured using the DEA and 
ratio approaches. The quadriform approach is not used in measuring school efficiency since 
financial data are not available at that level. The availability of data on staff personnel per pupil 
enabled the other two approaches to measuring efficiency. 

Standard Score Charts 



There are many bases for making comparisons, each having its advantages and disadvantages. 
Comparing ranks within a group of schools or districts is one approach. A basic difficulty with 
ranks is that the differences in the raw scores of measured attributes between ranks at the 
extremes are likely to be large when compared to those at the center of a distribution, i. e., there 
may be little difference of consequence from a policy perspective between ranks of 40 and 60 in a 
distribution of a hundred cases whereas the difference between ranks of 10 and 1 may be 



considerable The problem is reduced only marginally when percentiles, deciles or quartiles are 
used. These statistics also have the additional disadvantage that they cannot be added, subtracted 
or averaged. Also, because of the unequal intervals, these statistics are not very useful when 
comparisons are made among a number of attributes. 

To facilitate comparisons on a given attribute (e g., high school achievement), and across 
attributes, we have converted the “raw” scores of each distribution to a common metric with the 
mean equal to zero (0) and the standard deviation equal to one (1) under the assumption that each 
attribute approximates a distribution bounded by a “normal curve.” The conversion results in 
what is called a standard score. Standard scores have an equal interval scale (one standard 
deviation) and can be aggregated and averaged. 

All of the benchmarking charts prepared in this study use standard scores. Charts were 
developed for analyzing student achievement on fifteen performance measures at the school level 
for the academic years 1993-94 through 1995-96. One chart each was developed for showing 
aggregate student achievement and school/district effect at the school and district levels 
respectively for the academic year 1995-96. Six standard score charts were developed to 
facilitate assessing district fiscal policy for the academic year 1995-96. 

Efficiency Analysis ^ 

As with most studies of relationships between student outcomes and expenditures, there was no 
statistically significant correlation between school district effect and expenditure (r = .15; level of 
significance, .15). In the quadriform analysis of efficiency, school districts with a school district 
effect on student achievement more than one standard deviation above the mean (the top 1 7 per 
cent of the districts) ranged in expenditure from $3,849 per pupil unit (Panama) to $7,987 
(Chautauqua before merger with Mayville), both in Chautauqua County. School districts with a 
school district effect on student achievement more than one standard deviation below the mean 
(the lowest 17 per cent of the districts) ranged in expenditure from $3,631 per pupil unit 
(Falconer) to $5,892 (Maryvale). The difference in expenditure per pupil unit between Falconer 
and Panama (both in Chautauqua County) is less than $200, yet Panama’s district effect on 
student achievement is more than two standard deviations higher than Falconer’s. 

Looking at school effect from the standpoint of expenditures, school districts spending 
more than one standard deviation below the mean range in effect from 1.36 standard deviations 
above the mean for Akron in Erie County to 2.76 and 2.55 standard deviations below the mean 
for the Kendall and Albion school districts in Orleans Courity. The one district spending more 
than two standard deviations below the mean (Falconer) had a school effect more than one 
standard deviation below the mean. Districts spending more than one standard deviation above 
the mean range in effectiveness from 1.94 standard deviations below the mean for Niagara Falls to 
1.46 above the mean for Chautauqua (before its merger with Mayville). The two school districts 
spending more than two standard deviations above the mean (Chautauqua and Barker in Niagara 
County) had school effects more than one standard deviation above the mean. 
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There is not perfect agreement among the four efficiency models studied as to which districts are 
deemed efficient and which are deemed inefficient. This is not wholly unexpected since the DEA 
technical efficiency model measures a different concept of efficiency (technical) than do the other 
models (economic). Ten districts were identified as being consistently in the most efficient 
quartile by all four models (Akron, Frontier, Hinsdale, Letchworth, Lyndonville, Newfane, 
Panama, Pioneer, Portville, and Springville). Four other districts (Allegany/Limestone, Angelica, 
Olean and Randolph) were identified as being consistently in the most efficient quartile by the 
three economic models. Nine districts were identified as being in the least efficient quartile by all 
four models. Six others were consistently identified as being in the least efficient quartile by the 
three economic efficiency models. 

All of the consistently efficient districts were above average in school effect with an 
average standard score of +.79. On the other hand, all of their expenditures per pupil need unit 
were below average with their average being one standard deviation below the mean. The school 
effect score for all of the consistently inefficient schools was below average with an average 
standard score for the group of -.74. All of their expenditures per pupil need unit were above 
average for the sample with the average being one standard deviation above the mean. 

School districts in the Buffalo metropolitan area are over-represented in the most efficient 
quartile according to the technical efficiency model; but they are under-represented in the top 
quartile according to all three economic efficiency models. Rural districts tend to have lower 
pupil to teacher ratios than do urban and suburban districts, ranking them lower on technical 
efficiency. On the other hand, rural districts pay less for their teachers and other staff, giving 
them an advantage on economic efficiency measures. 

At the school level, thirty six elementary schools, thirteen middle schools, five 
junior/senior high schools and ten high schools were classified as efficient by both models. Of 
these schools, three were located in Buffalo, seventeen in small cities, forty in the suburban/rural 
areas, and four in the rural areas. These represent 5%, 22%, 18% and 10%, respectively, of the 
total schools in each of these categories. Furthermore, forty-five elementary schools, fifteen 
middle schools, ten junior/senior high schools, and ten high schools were classified as inefficient 
by both models. Of these, twenty-eight were located in Buffalo, sixteen in small cities, thirty-one 
in the suburban/rural areas, and five in the rural areas, representing 46%, 21%, 14% and 13% of 
the schools in each of these categories. 

Conclusions 



The information generally available for making school district comparisons has left much of the 
story untold. Some of the analyses that have been done and published by the mass media may 
serve a symbolic purpose, but they are of little strategic value in helping people to make beneficial 
changes in our educational systems. This study has added two dimensions, school district effect 
and school district efficiency, to the one normally considered, student achievement. In the 
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process, we have highlighted the differences in challenges that school districts face. When more 
dimensions are considered, it is readily apparent that many school districts, not normally 
recognized for their effectiveness, are making important contributions to the intellectual 
development of their students. The contribution of other districts commonly recognized as 
outstanding are put into perspective. This does not, however, relieve schools and districts that 
have high school effects but whose students are still failing to meet established achievement 
standards of the responsibility for bringing those students up to standard; it just further illustrates 
the magnitude of the task. This study has shown that more spending on education is not the sole 
means to better results; money has to be spent wisely in order to achieve the results desired. 

This is where the concept of efficiency becomes very important. 

To assist in the difficult task of assessing school district fiscal policy, we have developed a 
series of charts to facilitate the process of benchmarking through making external comparisons. 
We have also developed a series of achievement charts that enable schools to visualize their actual 
contribution to student achievement and to make comparisons of student achievement with 
selected reference groups. 

Considerable sophistication has been developed in measuring the equity of resource 
allocation among and within school districts! As public concern over efficiency of school 
operations grows, more sophistication needs to be gained in defining and measuring that concept. 
We hope that this exercise has contributed to the general understanding of the forces contributing 
to student achievement, to analyzing efficiency of school operations, and to the need for further 
research on the topic. 

Further, if an assessment is made that these statistics and charts are useful to school policy 
makers and to the general public in better understanding the decisions that have to be made, and 
in ultimately making better decisions, then judgements need to be made as to whether this type of 
analysis should be continued on a regular basis and how. 
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Benchmarking: A Study of School and School District Effect and Efficiency 

Context and Purposes 

That which cannot be measured cannot be controlled. This understanding is what is driving the 
standards movement. In the past, it has been common practice for educators to claim success on 
very little tangible evidence other than the intuitive feeling of those professionals involved in 
designing and implementing a program. Now higher levels of govemment-the state in particular 
and the public in general-are asking, “How do you know? What evidence do you have that 
supports your perceptions and opinions?” 

Accountability has become a principal component of educational reform initiatives in the 
United States (Ginsberg and Berry, 1998). For decades, financial accountability has been 
common practice whereby school district records of revenues and expenditures are subjected to 
external audit and the results are made available as a matter of public record. Since the release of 
A Nation at Risk (National Commission on Excellence in Education) in 1983, the concept of 
accountability has been extended to include pupil achievement with the establishment of state 
academic standards, uniform state testing programs, and published results. The two 
accountability systems are typically maintained and analyzed separately. 

Benchmarking is a form of assessment that establishes external standards to which internal 
processes may be compared. The New York State School Report Card provides a vehicle for 
benchmarking with respect to student achievement. In this study, we have developed additional 
tools for making external comparisons with respect to achievement and have added tools for 
assessing fiscal policy and efficiency, i.e., comparing outcomes (achievement) with the amount of 
resources consumed in the schooling process. 

One of the most articulate champions of management (and decision making) by fact was 
W. Edwards Deming (1986). The Deming principles were originally applied to the management 
of business organizations, first in Japan and subsequently in the United States. Now they are 
being applied to the organization and management of schools, school districts, and other 
educational organizations (Frazier, 1997; Schmoker & Wilson, 1993). Deming contended that 
collecting and analyzing data on what is most important to an organization is essential for 
improvement, for validating efforts, and for refining strategies. What is more important to 
schools, to parents, and to the public in general than student outcomes? Equally important to the 
taxpayers who pay for the personnel, facilities, equipment and supplies used in developing those 
outcomes is the assurance that those resources are being applied efficiently and are not being 
wasted in any way. 

The Consortium on Renewing Education (CORE) includes as a major part of its strategy 
for doubling academic achievement of school children in the United States by the year 2020 the 
making of comparative analyses across states and local districts. CORE (1998) contends that 
comparison and competition provide a major “engine” for implementing school reform. “It 
motivates state and local officials to seek new avenues for improvement. No one, and almost no 
institution, enjoys unfavorable comparisons” (P. 68). 
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2 School and School District Efficiency 

The first purpose of this research has been to bring together the concepts of financial 
(inputs) and achievement (outputs) accountability, developing methodological procedures for 
studying the efficiency of schools and school districts that complement existing studies of 
effectiveness and fiscal prudence as independent concepts. The second purpose has been to 
develop a series of benchmarking charts that facilitate the efforts of policy makers at the school 
and district levels to analyze the effectiveness and efficiency of their operations. These charts 
assist in the quest of obtaining better results from the resources available by examining trends and 
making comparisons with reference schools and districts. In pursuit of these purposes, this study 
has developed: 

1. indices of student achievement at the school and district levels that 
aggregate the results of several outcome measures; 

2. indices of school and district effect on achievement that are independent of 
effects external to the school such as student home influences; 

3. indices of resource inputs into the schooling process; 

4. indices of school and school district efficiency; and, 

5. reporting devices that facilitate the evaluation of school and district 
policies using the indices developed. 

The public school enterprise consumes a lot of money-over $25 billion a year in New 
York State and over $300 billion nationally, a record amount even when accounting for inflation 
and student population growth. Despite these record expenditures, many schools and districts are 
failing to prepare their pupils as fully functioning citizens in today’s society. It is commonly 
thought that the failing districts and schools are failing because they lack the resources that are 
available to succeeding schools. This is not necessarily so. For over three decades, the 
relationships between school inputs and outcomes have been carefully studied with the startling 
conclusion that there is very little evidence showing that such relationships do exist once the 
family backgrounds of pupils are appropriately accounted for (Hanushek, 1986)'. 

This is not to say that all schools are not using their resources to achieve the greatest 
effect. During the 1980s there was the Effective Schools Movement that identified specific 
schools that had an unusually strong academic impact on their pupils (Brookover & Lezotte, 

1979; Edmonds, 1979); but, these were “outliers,” the exceptions and not the rule. A valiant 
effort to export their practices to less successful schools met with very limited success. 

There are challenges to the conclusions that there is no systematic relationship between 
school expenditures and student performance (Hedges, Laine, & Greenwald, 1994); but, even the 
challengers acknowledge that, “the results do not provide detailed information on the most 
educationally or economically efficient means to allocate existing and new dollars” (Hedges & 
Greenwald, 1996). All agree that we must develop a better understanding of how resources can 



'For a full review of this research and its implications see Chapter 13 in Swanson and King 

(1997). 
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be used to greater positive effect in specific circumstances. Hanushek (1996) concludes: 

The central issue in all policy discussions is usually not whether to spend more or less on 
school resources but how to get the most out of marginal expenditures. Nobody would 
advocate zero spending on schooling, as nobody would argue for infinite spending on 
schooling. The issue is getting productive uses from current and added spending. The 
existing evidence simply indicates that the typical school system today does not use 
resources well (at least if promoting student achievement is their purpose). It is 
tautological to say that we will get good performance if we spend the money wisely. 

Today the existing knowledge base does not ensure that any added funds will, on average, 
be spent wisely. That is true even if some schools may spend their funds wisely, (p. 69) 

Hanushek (1996), along with many other public analysts, believe that this situation exists 
because school personnel have had little at stake in improving student outcomes. In the past, 
there have been few incentives linking rewards to teacher and administrative performance; but the 
situation is changing rapidly, especially in New York State with the aggressive establishment of 
high performance standards for students and with the open publication of results through the 
School Report Cards. Commissioner of Education, Richard P. Mills has been quoted recently 
(Hendrie, 1999) as saying, “I’ve told them [local school officials] over and over again that there is 
a test with one question that they will face this year, and that is, ‘What did you get for the 
money?”’ 



Tightening the linkages between inputs and student performance is not going to be an easy 
task, especially when one is not certain what those linkages are. This study is intended to assist 
school districts in their appraisal of their policies concerning the application of resources to the 
schooling process and their success in accomplishing higher student achievement. New York 
State is a good venue for studying such policies because of the ready availability of pupil 
achievement information and Western New York, in particular, is a good place because of the 
ready availability of financial information in a user friendly format (Erie No. One Board of 
Cooperative Educational Services, annually). ^ 

New York State is unique among states in that it has offered continuously state criterion 
referenced examinations for over a century, the New York State Regents Examinations. In the 
early days, successful completion of these examinations resulted in the awarding of state issued 
certificates and diplomas that provided a common basis to employers and colleges for assessing 
achievement of students completing their primary and secondary education. At that time, many 
children received their primary education in thousands of poorly supervised one room 
schoolhouses and their secondary education in hundreds of very small high schools that varied 
greatly in quality. As school districts throughout the country became better organized and staffed 
with better trained teachers, other states eliminated state developed examinations and the issuing 
of state diplomas. New York State, alone, continued to offer the Regents examinations and 
diplomas for high school subjects, although the primary completion examinations and certificates 
were dropped with the raising of the compulsory attendance age to sixteen. 
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In 1966 the New York State Education Department initiated the Pupil Evaluation Project 
(PEP) that led to the development of a series of examinations assessing achievement in the 
elementary and middle grades. These examinations were intended to measure the 
accomplishments of schools and districts, not individual students. Reports were made available to 
schools, school districts and researchers, but not to the general public until 1987. 

Chapter 655 of the New York State Laws of 1987 required the Board of Regents and the 
State Education Department to submit an annual report to the Governor and the Legislature 
concerning, among other things, student achievement (as measured by the PEP tests and Regents 
examinations), student and teacher characteristics, and financial information. Data are reported 
by district, district type, ethnic group, etc. The report. New York: The State of Learning , is due 
January first of each year and usually becomes available to the public sometime in February. The 
report released in February of 1995 also published for the first time 1990 decennial census data 
for school districts which are useful in statistically controlling for variations among school districts 
in socio-economic characteristics that influence student achievement. These reports are available 
to the general public, but few avail themselves of them, although some enterprising newspaper 
reporters do massage the data into articles with eye-catching headlines. 

In 1996, the State Education Department issued the first School Report Cards. These 
report cards focus on academic achievement, although they also give information on attendance, 
suspensions, school organization, pupil demographics and, district total and approved operating 
expenditures per pupil. Average achievement is reported for the school, district, schools with 
similar demographic characteristics, and all public schools in the state. For the first time, a 
concerted effort was made to assure that the information was made available to the general public 
by requiring school districts to develop a school report card, share it with newspapers, attach it to 
the school budget, and make it publicly available. The State Education Department publicly 
releases its information on each school and district and makes it available on the Department’s 
World Wide Web site. 

Following the national trend, the New York State Board of Regents authorized in April 
1994 the development of academic standards in English language arts, mathematics, science, and 
social studies for children and youth enrolled in elementary and secondary schools and instruments 
for assessing the achievement of those standards. While much attention has been directed toward 
the upgrading of high school Regents examinations, the state tests for elementary and middle 
school children are being revised to reflect the new learning standards and to align them with the 
high school examinations. 

The situation is dynamic. New examinations are being developed and phased in through 
the year 2001 . Additional information about the performance of students with disabilities and 
limited English proficiency students is now being published. New reporting procedures have been 
developed whereby, beginning with the 1998-99 school year, assessment results by individual 
student (rather than school and district aggregates) will be transmitted electronically to the State 
Education Department in Albany using an enhanced version of the Local Education Agency 
Program (LEAP) reporting system previously used by schools receiving Title I and other special 
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funding. These changes will enable policy analyses of much greater detail and sophistication than 
is now possible. 

While this study used data available for the school years 1993-94 through 1995-96, the 
procedures and reporting devices can be adapted and updated as new information becomes 
available. Achievement data were obtained from the 1 997 New York State School Report Card 
data base. Financial data for the 1995-96 school year were derived from Financing Our Schools- 
1997, prepared by the Erie County No. One Board of Cooperative Educational Services. The 
cases were limited to the 101 school districts and their schools in the western region of the state 
containing the two county Buffalo metropolitan area and six rural counties, three of whom 
contain small cities. The region is geographically about the size of the state of Connecticut and 
contains approximately 1.5 million people. 

Two types of outcome measures were developed: average student achievement and 
average school and district effect. Average student achievement is measured by four factor scores 
at the district level and six average scores at the school level. Both sets of scores combine results 
of state examinations in reading, writing, mathematics, science and social studies given in grades 
3, 4, 5, 6 and 8 and for fifteen Regents examinations given in high school. School district effect 
is estimated through a multiple regression procedure that statistically removes co-variation of 
family and environmental characteristics from the student achievement scores. Input measures 
consist of expenditures (adjusted for within region variations in teacher costs) per pupil units 
(adjusted for variation in pupil characteristics) and pupil staff ratios. School district efficiency is 
measured using a linear programming technique known as data envelopment analysis (DEA), a 
modified quadriform approach, and a ratio approach. School efficiency is measured using the 
DEA and ratio approaches. The quadriform approach is not used in measuring school efficiency 
since financial data are not available at that level. The availability of data on staff personnel per 
pupil enabled the other two approaches to measuring efficiency. 

Explanation of how aggregate student outcome measures were developed is presented 
first followed by discussions of the schooling input and efficiency measures. The fourth section of 
this report presents a series of benchmarking type charts that sharpen the ability of policy makers 
to evaluate the performance and fiscal policies of their districts and schools in comparison with 
the other districts and schools in the region. 

School and District Output Measures 
Average Aggre£ate Student Achievement 

Pupil achievement is popularly considered to be an indicator of how well students are equipped 
for entrance into institutions of higher learning and into employment. New York State reports 
annually the average results at various levels of proficiency by school and by district for tests 
given in grades three through twelve. Because of the large number of scores reported, it is 
difficult to get a general sense of how well a school or district is doing without further analysis. 
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To construct global measures of student achievement at the district level, we subjected the 
results of the test batteries to a statistical procedure known as factor analysis and generated four 
factor scores reflecting achievement at grade 3, grades 4 and 5, grade 6, and high school. The 
third grade achievement factor consists of the percent of students scoring above each of four state 
reference points on examinations in reading, and scoring above the state’s minimum reference 
point in mathematics— all weighted to maximize the total variance explained. The fourth/fifth 
grade achievement factor consists of the average score on the fourth grade science test and the 
percent scoring above the state minimum reference point on the fifth grade writing test. The sixth 
grade achievement factor consists of average scores similar to those for third grade plus the 
average score on a social studies test. The high school achievement factor consists of percent of 
enrollment for an appropriate grade cohort passing and the percent achieving mastery on Regents 
examinations in fifteen subject areas— each weighted to maximize the total variance explained. 

It was not possible to do a factor analysis at the school level because the great variation in 
grade configurations among buildings would result in a missing data problem that would eliminate 
most schools from the analysis. Instead, we computed an average of the standardizes scores of 
tests for each elementary grade in which tests were given. Three averages were developed for 
each high school, percent passing and percent achieving mastery on Regents examinations and per 
cent passing the occupational education Regents. 

Standard Score Charts 



There are many bases for making comparisons, each having its advantages and disadvantages. 
Comparing ranks within a group of schools or districts is one approach. A basic difficulty with 
ranks is that the differences in the raw scores of measured attributes between ranks at the 
extremes are likely to be large when compared to those at the center of a distribution, i. e., there 
may be little difference of consequence from a policy perspective between ranks of 40 and 60 in a 
distribution of a hundred cases whereas the difference between ranks of 10 and 1 may be 
considerable. The problem is reduced only marginally when percentiles, deciles or quartiles are 
used. These statistics also have the additional disadvantage that they cannot be added, subtracted 
or averaged. Also, because of the unequal intervals, these statistics are not very useful when 
comparisons are made among a number of attributes. 

To facilitate comparisons on a given attribute (e g., high school achievement), and across 
attributes, we have converted the “raw” scores of each distribution to a common metric with the 
mean equal to zero (0) and the standard deviation equal to one (1) under the assumption that each 
attribute approximates a distribution bounded by a “normal curve.” The conversion results in 
what is called a standard score. Standard scores have an equal interval scale (one standard 
deviation) and can be aggregated and averaged. 

A normal curve is a symmetrical, bell shaped curve as illustrated in Figure 1 . Its 
importance stems from the fact that chance events in large numbers tend to distribute themselves 
within the boundaries of the curve. For this reason, it has been of great interest to mathematicians 
and astronomers for over two centuries and to statisticians for most of this century. The mean of 
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the distribution bisects the curve with half of the cases (and area) being to its left and half to its 
right. A standard deviation can be conceived as a length along the base line of the curve from the 
mean out to the right or to the left to the point where the curve inflects. In a normal distribution, 
approximately two-thirds of the cases fall between one standard deviation above the mean and 
one standard deviation below the mean (i. e., between standard scores of +1 and -1). Ninety-five 
per cent of all cases will be within two standard deviations of the mean or between standard 
scores of +2 and -2. 

Figures 2 and 3, respectively, show the standard score charts for district and school 
achievement measures described in the previous section. The normal distribution scale is shown 
on the left of each chart. There is a vertical bar for each score reported with a designation for 
each standard deviation above or below the mean. The highest marking on a bar represents the 
highest score in the distribution; the lowest marking represents the lowest score in the 
distribution. The area between one standard deviation above the mean (+1) and one standard 
deviation below the mean (-1) has been shaded. Two-thirds of all cases (districts or schools) will 
have scores in this range and variation within this range is not highly significant for policy 
evaluation purposes. Schools or districts scoring above or below the shaded areas are outliers. 

No distribution matches the normal distribution perfectly; some are skewed above the mean and 
some are skewed below the mean. 

Averaee A2greeate School and District Effect 

School and school district effectiveness is frequently judged erroneously on the basis of 
unadjusted student achievement. At least thirty years of research into the matter strongly suggest 
otherwise. Research studies that explain differences in student achievement have shown it to be a 
function of many forces. The unique influence of formal schooling is relatively small in 
comparison with other forces such as education level of parents, family environment, and peer 
group interaction'. The New York State Education Department (1997) has identified four 
indicators associated with poor school performance: minority racial/ethnic group identity, living in 
a poverty household, having a poorly educated mother, and having a non-English background. 

The findings of a recent study by the RAND Corp. (Grissmer, Kirby, Berends, and 
Williamson, 1994) is consistent with earlier findings. The RAND study identified the most 
important family characteristic influencing student performance in school as parents' education. 
Income, family size, and mother's age at child's birth were modestly significant. Whether the 
mother worked or not had a negligible effect after accounting for other family factors. Single- 
parent status by itself was not significant although single-parenthood is highly correlated with 
other factors that have negative effects on achievement such as low family income. The presence 



*An abbreviated list of citations include: Coleman, 1966; Mayeski, et al., 1972; Summers 
& Wolfe, 1975; Bridge, Judd, & Moock, 1979; Rutter, Maughan, Mortimore, & Ouston, 1979; 
Dreeben & Thomas, 1980; Madaus, Airasian, & Kellaghan, 1980; Hanushek, 1986, 1991, 
Ferguson, 1991; Hedges, Laine, & Greenwald, 1994; Sandowski, 1995. 
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Figure 2. Achievement Chart XVII: Aggregated Student Achievement and School Effect for 
Grades 3, 4/5, 6 and Regents Examinations for Districts (in standard scores) 




Figure 3. Achievement Chart XVI: Aggregated Student Achievement and School Effect for 
Grades 3, 4/5, 6, 8 and Regents Examinations for Schools (in standard scores) 
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of these factors affecting student achievement differ markedly among school and district 
populations, meaning that the work of some schools and school districts is assisted by favorable 
environmental factors and is frustrated by unfavorable environmental factors in others. The 
pernicious effects of poverty are compounded when poor students are highly concentrated 
(Kennedy, Jung & Orland, 1990; Abt Associates, 1993; Pelavin Associates, 1990). 

In total, school takes up only about 13 percent of the waking hours of a person's first 18 
years of life (Walberg, 1984). Children receive their initial instruction (particularly in language 
arts) in the home and in the community, albeit informally, and those children whose parents are 
well-educated usually come to school better prepared to function more efficiently in an 
environment that emphasizes abstract concepts than do children whose parents are less well 
educated. Schools composed of children who already have developed good learning and language 
skills can begin instruction at a more advanced level than can schools where most of the children 
enter with poor skills. The problem of poor entry level skills for many American schools is 
recognized by the first of the eight national goals for public education set by Congress in 1994 
with the Educate America Act: By the year 2000, all children in America will start school ready 
to learn. Throughout the schooling process, the work of the school is re-enforced for children 
living in language rich home environments, but not for other children, causing the achievement 
gap between the two groups to widen. The differences in population characteristics among 
schools and districts mean that some schools and districts must meet their instructional 
responsibilities in quite different ways than others do. 

Research in the United States on the impact of family socio-economic status (most 
importantly, education and income) on the achievement of children has been clouded by the issue 
of racial and ethnic group membership. Despite the ethnic focus, low socio-economic status has 
emerged as the dominating detracting factor from achievement with little, if any, effect being 
explained independently by minority group membership. This is not to deny that racial and 
cultural minority children experience discriminatory situations which have an additional negative 
impact on the development of self-concept and realistic aspirations and expectations. Some social 
scientists refer to the treatment of racial and cultural minorities in the United States as functioning 
more like'a caste system than socio-economic differentiation (Ogbu, 1978; Brown, 1990). 

Socio-economic characteristics are only proxies for interactions within families and society 
which tend to be related to socio-economic status. Home environment indicators predict 
academic learning twice as well as socio-economic status of families (Walberg, 1984), but they 
are much more difficult to measure for research purposes-especially on a large scale— and data on 
home environments are not readily available. Correlational and status studies are useful in 
pointing out the overall impact of socio-economic status on pupil achievement, but they do little 
to advance our understanding of how the effect is transmitted or what educators can do to 
intervene. 

The inter-relationships between environment and student achievement are too complex to 
be explained through the lens of a single discipline. Four main perspectives characterize the 
literature on school performance of children from lower SES families and racial and cultural 
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minorities: the cultural continuity/discontinuity approach, the secondary cultural continuity 
approach, cultural reproduction theory, and the culture and cognition approach (Emihovich, 

1994). Poor academic achievement by such children is attributed in these theories to a variety of 
factors including: 

• differences between home and school in interactional, linguistic, and cognitive styles; 

• effects of macroeconomic and social conditions, especially labor market forces and 
minority groups' beliefs about their access to employment and other social benefits; 

• family values concerning the importance of education, adherence to prevailing social 
norms, and allegiance to community welfare rather than individual gain; 

• the school's perceived role in reproducing the social order to maintain class and racial 
barriers to social mobility; 

• student resistance to learning behaviors expected by school authorities which would 
bestow upon the students identities which are stigmatized among their peers; and, 

• individual variations in performance as a function of culturally influenced cognitive 
capacities. 

Several of the above perspectives have been unified through the concept of multiple literacies. 

[E]ach literacy is embedded within particular culturally organized settings, shaped by 
children's early experiences in the home and community environments, and influenced or 
, modified by alternative literacies children encounter daily in schools and other social 
settings. In short, for children to be successful in school and society, they need to master 
a broad range of literacy competencies, almost in the sense of being multilingual, to cope 
with the diversity they can expect to encounter in written and oral formats across a wide 
array of situations. (Emihovich, 1994, p. 1231) 

Research clearly shows that language and cultural differences in students' lives are 
interwoven with economic and social conditions that facilitate or impede knowledge acquisition. 
This bonded relationship must be taken into account in designing instructional strategies for 
children. It must also be factored into any assessment of school and district performance. 

To estimate the effect of school districts on student achievement, we have used residual 
scores derived from multiple-regression analysis with the measures of student achievement 
previously described serving as dependent variables and family, school and district characteristics 
serving as independent variables. The district analysis used a census poverty index, combined 
wealth index (a measure used in calculating New York State aid to school districts consisting of 
an average of the ratios of per pupil property value and personal income to the state averages of 
those statistics), percent of children qualifying for free lunch, percent children with limited English 
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proficiency, and district classification (e. g., urban, suburban, rural) as control variables. For the 
school level analysis, percent of children qualifying for free lunch, percent children with limited 
English proficiency, and district classification served as control variables. 

The residual scores partial out the wealth effect from the student achievement scores 
producing what we call a measure of school effect. The nature of the correction at the school 
district level is shown in Figure 4 using the standard score chart shown in Figure 2. Figure 4 
shows the school district effect adjustment (dash line) and unadjusted student achievement (solid 
line) for three school districts in the region. Buffalo is a large central city school district with a 
sizeable proportion of its population in poverty; thus, the school effect scores indicate a greater 
school effect than student achievement scores alone would suggest. Williamsville is an affluent 
suburban district; statistically controlling for the effects of socio-economic status of the district on 
student achievement makes the school effect scores less than student achievement scores alone 
would suggest. Batavia is a small city school district of average socio-economic characteristics 
for the sample; because of its average demographic characteristics, the school effect scores are 
virtually identical to the student achievement scores. Interpretation of standard score profile 
charts is discussed in the “Policy Analysis Tools” section of this report. 

The nature of the correction at the school level is shown in Figure 5 using the standard 
score chart shown in Figure 3. The school effect adjustment (dash line) and unadjusted student 
achievement (solid line) are shown for four schools in the Buffalo City School District. City 
Honors is a magnet secondary school (grades 5-12) that screens applicants using an entrance 
examination. As a result, its student achievement is high (the highest in the region) and the socio- 
economic status of its students is relatively high for the region and very high for the city school 
district; this yields a school effect that is generally lower than the school’s unadjusted student 
achievement— especially on the Regents examinations. Grover Cleveland is a neighborhood high 
school serving a mostly poor population, many of whom are non-English speaking. Its student 
achievement scores are nearly two standard deviations below the mean of the sample for passing 
and mastering the Regents examinations; but, its school effect scores are slightly above the sample 
mean. Although the Futures Academy is a magnet school serving grades K-8, its pupil 
characteristics are similar to those for the district as a whole; its student achievement is near 
average for the sample and its school effect is largely above average. Hamlin Park elementary 
school is a low achieving neighborhood school serving a generally poor population; adjusting the 
student achievement scores for socio-economic factors moves the school effect scores only 
slightly higher. The higher student achievement scores for Futures Academy than for Hamlin Park 
may reflect family characteristics and attitudes not measured but made operational through the 
district’s magnet school choice policy (Adler, Petch, & Tweedie, 1989; Witte, 1993, Martinez, 
Thomas, & Kemerer, 1994). Or, the differences may reflect more effective instructional or other 
strategies at the Academy. 
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Figure 4. Achievement Chart XVII: Aggregated Student Achievement and School Effect for 
Grades 3, 4/5, 6 and Regents Examinations for Districts (in standard scores) 
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School and District Input Measures 

Two types of inputs into the schooling process were used, numbers of personnel (teachers, 
other professionals, and para*professionals) and financial. Both inputs were adjusted for 
differences in sizes of schools and districts and the characteristics of their pupils by dividing by a 
measure of pupil educational need (i.e., teachers per 100 educational need unit or dollars per 
need unit). The financial measures in the efficiency analysis were also subjected to a correction 
for variations in the cost of teachers. The nature of these corrections is described below. 

Measuring Pupils’ Educational Need 

The scope of the aggregate pupil educational need of a school or district is crudely expressed in 
terms of the number of students served. This measure is closely tied to the size Of the teaching 
force, the sizes of programs and facilities, and ultimately, to the amount of money required. But 
resource requirements are not uniform among all pupils; some receive services that others do not. 
In an earlier section, we pointed out that how resource utilization is linked to student achievement 
is not known; never-the-less, state policy prescribes specific programs for pupils with certain 
characteristics, and sometimes sets staffing standards for those programs. Examples of this would 
be programs for pupils classified with some sort of disability and programs for the gifted and 
talented. Also, state policy recognizes common practice. For example, in several state aid 
formulas, a secondary pupil is treated as 1.25 elementary pupils because most school districts 
spend about 25 per cent more per pupil on their secondary school programs than they do on their 
elementary school programs. Ironically, there is considerable research suggesting that higher 
expenditures at the elementary level are more effective in terms of their impact on student 
achievement (e.g., Hitzges, 1988; Wenglinsky, 1998). 

The net effect of these state policies is to make the operations for some schools and 
districts more expensive than for others depending on the characteristics of the pupils served. 
Descriptions of per pupil expenditures that do not account for differences among program costs 
substantially overstates the resources reaching non-classified students (Lankford & Wyckoff, 

1999), making districts with high proportions of classified pupils appear unduly inefficient 
(Reschovsky & Imazeki, 1998; Parrish & Hikido, 1998). To compensate for these differences, we 
divided resource inputs by a measure of pupil educational need that reflects state aid policy. We 
began with the Total Aidable Pupil Unit (TAPU) used in several state aid formulas. TAPU is 
calculated by applying the following weightings to the average daily attendance of pupils with 
specified characteristics: 



'/2 day Kindergarten (K) .50 

Full day attendance, grades K-6 1 .00 

Grades K-6 Pupils with Special Educational Needs (PSEN)^ 1 .25 

Full day attendance equivalent, grades 7-12 1.25 



’PSEN reflects the proportion of pupils achieving below the State reference point on the 
PEP achievement tests administered in 1984-85 and 1985-86. 
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Grades 7-12 PSEN 1.50 

Summer School .12 

The above weightings do not reflect the additional costs of program accommodations for pupils 
designated to receive special educational services. To estimate the impact of these costs, .8 times 
the number of pupils so classified was added to the total TAPU. This weighting was based on an 
analysis of the ratio of costs of educational programs for the classified disabled compared to 
regular programs in Western New York (1.8). The resulting statistics are referred to 
subsequently as pupil need or EDNEED. 

Teacher Cost Index (TCI) 

There is a growing recognition of the importance of adjusting for cost variations among localities 
in making comparative studies of school costs (Chambers, 1996; Duncombe & Yinger, 1998; 
Parrish & Hikido, 1998; Reschovsky & Imazeki, 1998; Taylor, 1998). While we were aware of 
these studies at the beginning of this analysis, we did not believe that cost adjustments would be 
necessary within this relatively homogeneous economic region. As the analysis progressed, 
however, we discovered that a strong efficiency bias was developing against the school districts in 
the two county Buffalo metropolitan area due to their higher teacher costs. Subsequently, we did 
make cost adjustments for the efficiency analysis portions of this study and the reasonableness of 
the results improved. Cost adjustments were not made in the development of the financial 
standard score charts to be described in the “Policy Analysis Tools” section. 

We used Chambers’ and Fowler’s (1995) Teacher Cost Index (TCI) for our corrections. 
The index is based on the fact that approximately 80 per cent of educational costs are for 
personnel and that teachers account for most of this; further, the assumption is made that 
variations in other personnel costs will reflect variation in teacher costs. The TCI simulates the 
variations in teacher salaries resulting from only those factors affecting the supply of comparable 
teachers across geographic locations. Stated another way, the TCI is intended to measure 
variations in the price of teachers (and by implication, other personnel) which are beyond the 
control of local decision makers. 

While indices are available at the district level, we calculated an average adjustment for all 
districts in a county under the assumption that within county variations were more likely to be a 
function of school district policy (and thus a possible contributor to efficiency/inefficiency) than a 
function of structural factors beyond the control of the district. The adjustments by county are as 
follows: 

County Adjustment 



Allegany .9152 

Cattaraugus .9253 

Chautauqua .9622 

Erie 1.0740 
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Genessee 

Niagara 

Orleans 

Wyoming 



.9437 

1.0110 

1.0276 

.9527 



According to these indices, Erie County school districts face the highest costs, 7.4 per cent above 
the national average, and 15.9 per cent above the lowest cost county in the sample, Allegany. 

There is some evidence that the adjustment may be too small for the Buffalo City School 
District. In a study of New York State school districts, Duncombe and Yinger (1998) found 
considerable variation in costs per pupil required to achieve a given performance level. With 
1.0000 representing the state average cost, the index for Buffalo, Rochester, and Syracuse was 
1.7550 meaning that, for the three districts to achieve a given performance level, 75.5 per cent 
more cost would be incurred than for the average New York State district. Their indices for 
upstate rural areas were .9780, small cities 1.0650, and suburbs .9090. They found the most 
important factors influencing educational costs were: teachers salaries, enrollment, per cent of 
children in poverty, per cent female-headed households, per cent children with severe handicaps, 
per cent of students with limited English proficiency, and population density. The Duncombe and 
Yinger cost corrections represent a combination of the TCI adjustment and the pupil educational 
need adjustment discussed in the previous section and are, therefore, not directly comparable to 
the TCI correction. Chambers (1996) found that, nationally, urban area school costs ran about 15 
per cent above those incurred in rural areas. 



Efficiency is the ratio of outputs to inputs. Efficiency is increased by increasing the probability of 
realizing targeted outcomes secured from available resources or by maintaining a given level of 
outcomes while using fewer resources. In the education lexicon, efficiency concerns are 
expressed in terms of accountability and standards. 

While equity in the allocation of educational resources was a primary concern of policy 
makers in the 1960s and 1970s, the impact of a global economy has brought efficiency to the 
forefront in both the public and private sectors. Now they seek schools that are equitable and 
efficient. Over the years, policy analysts have gained a considerable amount of sophistication in 
analyzing public policy and performance for equity but are only beginning a quest for operational 
definitions and analytical techniques that will permit the study of efficiency in educational 
operations (Anderson, 1996). 

Most educators do not commonly use the concept of efficiency in analyzing resource 
allocation problems. As a result, they are being criticized currently for taking schooling inputs 
(pupils, teachers, buildings, classrooms, teaching supplies and equipment, heat, light, etc.) pretty 
much for granted-almost as a civil right of students and the profession. In assessing the 
shortcomings of the schools, educators tend to examine only curricular matters (technical 
efficiency) and not the costs associated with possible alternative solutions (economic efficiency). 
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This leads to a practice of “adding on” new programs to an existing set of programs rather than 
considering the restructuring of the entire educational enterprise. This conservative approach has 
made the educational system highly fragmented, very labor intensive and, in the view of some, 
overly expensive. 

Schools are institutions functioning largely in the public sector. They therefore do not 
face the stringent discipline for efficient operations imposed by the market on institutions 
operating in the private sector (Benson, 1978; Guthrie et al., 1988). Governments in capitalistic 
countries tend to deal with issues in which economic concerns are not overriding. As government 
activities have grown to consume over 40% of the gross national product, however, the public is 
forcing government officials-including school oflficials--to become more cognizant of the concept 
of economic efficieticy. 

Hanushek (1986, p. 1 166) defined economic efficiency as “the correct share of input mix 
given the prices of inputs and the production function.” Production function is defined as the 
causal relationship between inputs and outcomes; thus it is what goes on in schools and 
classrooms that directly affects the minds of students as measured by tests and other outcome 
measures. There is a distinction between economic efficiency and technical efficiency. The latter 
considers only the process of combining inputs to produce outcomes; as noted above, this is the 
usual focus of professional educators. Technical efficiency does not take into account the cost of 
inputs as does economic efficiency. Both types of efficiency are important in evaluating 
educational policy. 

Organizations can achieve efficiency in many ways. In order to be relatively efficient, a 
decision making unit (e. g., schools or districts) may have higher output levels than other units 
with the same level of inputs; it may even have output levels below average provided that the 
input levels are considerably lower than those of other decision making units. In education, we are 
not only concerned with the efficiency measure, but also with district effectiveness, that is, how 
well the district has met its educational objectives. While some minimum level of effectiveness 
should be an integral part of efficiency, it is possible for effective organizations to be inefficient by 
using an excessive amount of resources relative to other equally effective organizations. 
Furthermore, efficient organizations may be only moderately effective. In considering the 
relationship between efficiency and effectiveness, a number of possibilities arise. 

Efficient Districts 



Dealing first with efficient districts, the following scenarios are possible. 

• First, there are the efficient districts which are characterized by above average 

performance in terms of outputs. Such districts could be considered "star" performers and 
could serve as models for other districts attempting to improve their performance. Such 
districts may span the expenditure range, indicating that high expenditures and high 
efficiency are not mutually exclusive. While most districts strive to be "star" performers, 
not all efficient districts fall into this category. 
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• There are also efficient districts whose performance is below average; but since their 
expenditures are relatively lower, they are rated as efficient. These districts seem to be 
quite effective at controlling costs. The operational emphasis for these districts should be 
on improving performance, even if costs were to increase. For such districts, there may 
also be concerns regarding the adequacy of resources. Their performance is usually 
relatively good given the resources available; however, these districts may not have 
adequate resources to enable them to improve performance. Given the efficiency of these 
districts, it would be worthwhile to determine whether additional resources could be 
generated in order to improve student achievement. On the other hand, these districts may 
be able to purchase adequate educational resources due to favorable cost factors in the 
district; this seems to be especially likely for many rural school districts. 

Inefficient Districts 



With respect to districts evaluated as relatively inefficient we have: a) districts whose performance 
is above average, but, whose expenditures are very high compared with other districts; b) districts 
that are performing poorly and that have high expenditures; and c) districts that are performing 
very poorly and that have below average expenditures but these expenditures are high in relation 
to their performance indicators. The inefficiency of the districts in these categories may be due to 
a number of reasons including: 

• poor cost control; 

• administrators and/or teachers who are less skilled than in other districts; 

• significant cost factors beyond the district's control; 

• the purchase of resources providing outputs/outcomes which are not being measured; 

• economically disadvantaged children who require higher expenditures in order to attain 
acceptable performance. 

Examples of expenditures that may not necessarily result in additional measured 
outputs/outcomes include those to provide a more aesthetic environment, for enhanced arts or 
athletic programs, an enriched curriculum, etc. While such expenditures may not necessarily 
increase the measured outputs, they are often expected and supported by taxpayers, particularly in 
wealthy suburban districts. The Amherst and Williamsville school districts may be examples of 
this situation. 

Districts that find themselves with economically disadvantaged children who require 
higher expenditures to provide compensatory educational services may receive additional funding 
to deal with disadvantaged children; however, the amounts received may be insufficient to 
provide all of the services required to compensate for their disadvantages. Reschovsky and 
Imazeki (1998), for example, found the appropriate correction to be 159 percent. Buffalo, 
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Niagara Falls, and other core cities in the sample may be examples of this situation. Our analysis 
takes this phenomenon into account through the use of the EDNEED measure described above. 

Another example of a cost factor, beyond a district’s control is teacher experience. The 
single salary schedule awards teachers for each additional year of experience (up to a designated 
maximum), but, there is little evidence that experienced teachers are linked to higher student 
academic achievement. Several of the districts in the region, especially first ring suburbs, have 
experienced dramatic declines in enrollments-some losing more than half of their peak pupil 
enrollments. This caused them to lay off many teachers during the 1970s and early 1980s 
(beginning with the least experienced because of union agreements and state law). As a result, 
these districts now have very senior (and expensive) staffs and tend to be identified as 
economically inefficient. The Kenmore and Maryvale school districts may be examples of this 
situation. In contrast, districts that are growing and adding new faculty have relatively low salary 
costs and are often identified as economically efficient. These districts tend to be second ring, and 
sometimes affluent, suburbs; Orchard Park and Clarence may be examples of this situation. In a 
few years, as the second ring suburban staffs mature, staffing costs will increase significantly for 
them while costs in the first ring will decline as their older staff members retire and are replaced 
by younger and less expensive ones. It is possible that the economic efficiency classification of 
both types of districts can change with no change in operations or effectiveness. The technical 
efficiency of the districts would not be affected by this phenomenon. 

Another example of cost differences that are not fully under the control of school districts 
is labor costs. Even within the eight county Western New York region, there are substantial 
differences in the going rate for teachers and support personnel, especially between the two 
county Buffalo SMSA and the six rural counties. We attempted to control for these differences in 
our economic efficiency analyses by adjusting expenditures by the Teacher Cost Index (TCI) 
(Chambers &. Fowler, 1995) as explained in a previous section. 

School District Efficiency 

Modified Quadriform Analysis . The quadriform type of analysis was initiated by 
Hickrod et al. (1989) and slightly modified by Anderson (1996). In this type of analysis, an 
adjustment for unalterable school characteristics, mostly related to socio-economic status, is made 
for both student achievement and expenditure. The adjustment for student achievement was 
similar to our school effect measure. We do not believe that a socio-economic adjustment for 
expenditure is appropriate because much of the higher expenditures of affluent districts can be 
attributed to the purchase of lifestyle accouterments beyond that needed to provide for basic 
educational programs of quality (Bigenwald, 1977). We did, however, adjust expenditures for 
variation within the region for the cost of teachers using the TCI index. We also adjusted 
expenditures for variation in pupil characteristics among districts related to program 
differentiation by using the EDNEED measure as previously described instead of unadjusted 
enrollment. 

The first level of analysis was merely to plot the 101 districts in the region according to 
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average district school effect (Y or vertical axis) and approved operating expenditure per 
EDNEED (X or horizontal axis) using standard scores for each measure. The measure of school 
effect was determined by averaging the standard scores of the four district level measures of 
school effect (grade 3, grades 4 and 5, grade 6, and high school). The resulting distribution of the 
districts is shown in Figure 6. All of the districts above the horizontal (X) axis are above average 
in school effect and all of the districts below the horizontal axis are below average in school 
effect. All of the districts to the left of the vertical (Y) axis are below average in expenditure per 
pupil; all of the districts to the right of the vertical axis are above average in expenditure per pupil. 
The intersection of the two axes represents being average on both expenditure and school effect 
measures. This modified quadriform analysis is a measure of economic efficiency. 

Districts in Quadrant I (upper left) are above average in school effect and below average 
in expenditure; school districts in this quadrant are the most efficient in the region according to 
this analysis. All but six of the districts in this quadrant are from outside the Buffalo metropolitan 
area (Erie and Niagara Counties), a slight under representation from the expected. Quadrant II 
(upper right) represents districts that are above average in school effect and are also above 
average in expenditures; there is a slight over representation of the high achieving, high 
expenditure metropolitan suburban districts in this quadrant. Quadrant III (lower left) represents 
districts that are below average in school effect and in expenditure. Quadrant IV represents 
districts that have a below average school effect and are high spending; these districts are the least 
efficient in the region according to this analysis. 

As with most studies of relationships between student outcomes and expenditures (see 
discussion in the “Context and Purposes” section at the beginning of this report), there was no 
statistically significant correlation between school district effect and expenditure (r = . 15; level of 
significance, .15). School districts with a school district effect on student achievement more than 
one standard deviation above the mean (the top 1 7 per cent of the districts) ranged in expenditure 
from $3,849 per pupil unit (Panama) to $7,987 (Chautauqua before merger with Mayville), both 
in Chautauqua County. School districts with a school district effect on student achievement more 
than one standard deviation below the mean (the lowest 17 per cent of the districts) ranged in 
expenditure from $3,631 per pupil unit (Falconer) to $5,892 (Maryvale). The difference in 
expenditure per pupil unit between Falconer and Panama (both in Chautauqua County) is less than 
$200, yet Panama’s district effect on student achievement is more than two standard deviations 
higher than Falconer’s. 

Looking at school effect from the standpoint of expenditures, school districts spending 
more than one standard deviation below the mean range in effect from 1.36 standard deviations 
above the mean for Akron in Erie County to 2.76 and 2.55 standard deviations below the mean 
for the Kendall and Albion school districts in Orleans County. The one district spending more 
than two standard deviations below the mean (Falconer) had a school effect more than one 
standard deviation below the mean. Districts spending more than one standard deviation above 
the mean range in effectiveness from 1 .94 standard deviations below the mean for Niagara Falls to 
1 .46 above the mean for Chautauqua (before its merger with Mayville). The two school districts 
spending more than two standard deviations above the mean (Chautauqua and Barker in Niagara 
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County) had school effects more than one standard deviation above the mean. 

Ratio Analysis . The ratio analysis adheres more closely to the classical definition of 
efficiency as the ratio of outputs to inputs than does the quadriform approach. In this application, 
the school effect measure is divided by the expenditure measure; both measures are the same as 
used in the modified quadriform analysis. Like the modified quadriform analysis, ratio analysis is 
a measure of economic efficiency. The boundaries of the quartiles of efficiency of this analysis are 
represented by the diagonal lines superimposed on the modified quadriform analysis as shown in 
Figure 6. The most efficient quartile of schools (furthest to the left) now includes a few very high 
performing school districts with expenditures slightly above average for the sample. It also 
includes a few school districts that are below average in school effect but much below average in 
expenditure per pupil. The districts in this quartile are disproportionately from outside the 
Buffalo metropolitan area. The least efficient quartile (furthest to the right) includes some 
districts above average in effectiveness, but much above average in expenditure. This quartile 
also includes some districts that have very low school effects but are only slightly below average 
in expenditure. 

Most of the districts in the second and third quartiles are near average in school effect and 
expenditure. The second quartile includes some very high achieving school districts with 
expenditures only slightly above average. The second quartile also includes a few districts with 
school effects slightly below average and expenditures considerably below average. The third 
quartile is made up mostly of districts whose school effect is slightly below average and whose 
expenditures are near average. A few high school effect districts with very high expenditures also 
fall into this quartile. 

Data Envelopment Analysis . The third procedure used to measure school district 
efficiency was a process known as data envelopment analysis (DEA). DEA uses linear 
programming concepts to determine the efficiency of an organization in using its resources in 
terms of outcomes achieved. Chames, Cooper and Rhodes (1978) described DEA as "a method 
for adjusting data to prescribed theoretical requirements such as optimal production surfaces, etc., 
prior to undertaking various statistical tests for public policy analysis" (p. 4). Figure 7 illustrates a 
hypothetical production possibilities curve for third and sixth grade achievement. Efficient 
districts lie on the curve, e.g.. A, B, C and D. District U is inefficient because it is inside the 
curve and not on its surface. To become efficient, district U would have to modify the use of its 
resources to more closely reflect practice found in the efficient districts. 

DEA is preferable to either ratio analysis or regression analysis in determining the 
efficiency of organizations that produce multiple outputs (see Bowlin, Chames, Cooper and 
Sherman, 1985; Banker, Conrad and Strauss, 1986; Sherman, 1986; Sexton, 1986; Chames, 
Cooper, Divine, Ruefli & Thomas, 1989; Seiford and Thrall, 1990). A number of advantages of 
the DEA approach are particularly relevant to education (see Sexton, 1986; and Sexton, Silkman 
and Hogan, 1986): 

1) DEA can simultaneously handle multiple inputs and outputs. 
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Figure 7. An Hypothetical Production Possibilities Curve for Third and Sixth Grade 

Achievement 




Third Grade Achievement 



2) DEA does not require parametric specification of the relationships between inputs and 
outcomes. 

3) Managerial strategies for improvement of inefficient decision-making units can be 
determined. Returns to scale information may also be available. 

4) DEA can be used to determine either technical or economic efficiency, if appropriate 
information is available. 

The primary limitation of DEA is that it is an extremal technique, and thus is sensitive to 
inaccurate data. It is also unable to provide measures of statistical association between inputs and 
outputs and this makes it more difficult to choose among different model specifications. 

It should be noted that DEA determines strict inefficiency. In some cases, good 
performance in some areas may result in a relatively high efficiency rating even if performance in 
other areas is mediocre. Given the input and output combinations of all the organizations, data 
envelopment analysis attempts to optimize the relative efficiency rating of an organization. In 
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determine relative efficiency, the DEA approach compares the district to all other districts in order 
to determine whether some weighted combination of those districts (subject to appropriate 
constraints) can outperform the district under consideration. If such a weighted combination can 
be determined, the district is deemed to be inefficient; if not, the district is regarded as efficient. 
Thus, if it is possible for an organization to be evaluated as efficient, the analysis will identify it as 
such. A district will be regarded as inefficient only if it truly is inefficient in relation to other 
districts. Thus, this technique errs toward efficiency, that is, placement of a district in the fourth 
quartile (least efficient) is most reliable and placement in one of the other quartiles is more likely 
to be subject to possible error. 

Two DEA analyses were done, one looking at technical efficiency (DEA Technical 
Efficiency Model) and the other looking at economic efficiency (DEA Economic Efficiency 
Model). For the technical efficiency model, the outputs are the four school effect scores; the 
input measures are the number of teachers, the number of other professionals, and the number of 
para-professionals per EDNEED (the same measure used in the modified quadriform and ratio 
analyses). The outputs for the economic efficiency model are the same as for the technical 
efficiency model; but the inputs are measured solely by the amount of the operating expenditure 
per EDNEED. 

The DEA economic efficiency model is most similar to the modified qaudriform and ratio 
analyses in concept and in results. With two exceptions, all of the districts in the most efficient 
quartile of the economic efficiency model are below average in expenditure and are 
disproportionately from outside the Buffalo metropolitan area. Most (16 out of 25) are above 
average in school effect. Those that are below average in school effect are well below average in 
expenditure. With two exceptions, all of the districts in the fourth quartile of efficiency as 
projected by this model are above average in expenditure and 17 out of 25 are below average in 
school effect. 

The DEA technical efficiency model is the only model investigated that attempts to 
measure technical efficiency. It has identified districts as efficient (first quartile) that fell in all four 
quartile rankings of the other models studied. Six of the twenty-five districts in the most efficient 
quartile are above average in expenditure; sixteen are above average in school effect. Thirteen are 
in the Buffalo metropolitan area, an over representation from that expected. Likewise, this model 
has identified districts as being inefficient (lowest quartile) that fell in all quartiles of efficiency as 
identified by the other models and includes all levels of school effect and expenditures and rural, 
suburban and urban districts. 

Discussion . There is not perfect agreement among the four models as to which districts 
are deemed efficient and which are deemed inefficient. This is not wholly unexpected since the 
DEA technical efficiency model measures a different concept of efficiency (technical) than do the 
other models (economic). Ten districts were identified as being consistently in the most efficient 
quartile by all four models (Akron, Frontier, Hinsdale, Letchworth, Lyndonville, Newfane, 

Panama, Pioneer, Portville, and Springville). Four other districts (Allegany/Limestone, 

Angelica, Olean and Randolph) were identified as being consistently in the most efficient quartile 



26 School and School District Efficiency 

by the three economic models. Nine districts were identified as being in the least efficient quartile 
by all four models. Six others were consistently identified as being in the least efficient quartile by 
the three economic efficiency models. 

All of the consistently efficient districts were above average in school effect with an 
average standard score of +.79. On the other hand, all of their expenditures per EDNEED were 
below average with their average being one standard deviation below the mean. The school effect 
score for all of the consistently inefficient schools was below average with an average standard 
score for the group of -.74. All of their expenditures per EDNEED were above average for the 
sample with the average being one standard deviation above the mean. 

It is interesting to note that Maryvale moved from the least efficient quartile as determined 
by the models measuring economic efficiency to the most efficient quartile as determined by the 
model measuring technical efficiency. Earlier, Maryvale was given as an example of a district 
with a very experienced (and expensive) staff as the result of a dramatic drop in pupil enrollment 
two decades ago. When only the cost of the staff is taken into account, Maryvale appears to be 
inefficient. When only the numbers of staff employed are taken into account and not their cost, 
Maryvale appears to be efficient. As Maryvale’s experienced staff retires and is replaced with less 
experienced persons, Maryvale’s economic efficiency scores are likely to improve dramatically. 
This raises interesting questions about the efficacy of the ways we commonly remunerate 
teachers. 

It is also interesting to note that school districts in the Buffalo metropolitan area are over- 
represented in the most efficient quartile according to the technical efficiency model. They had 
been under-represented in the top quartile according to all three economic efficiency models. 

Rural districts tend to have lower pupil to teacher ratios than do urban and suburban districts, 
ranking them lower on technical efficiency. On the other hand, rural districts pay less for their 
teachers and other staff, giving them an advantage on economic efficiency measures. 

Table 1 reports the Pearson correlation coefficients among the four efficiency models. All 
are positively correlated with one another, and all relationships are highly significant. The 
quadriform and ratio models are the most closely correlated followed by the DEA efficiency and 
ratio models. As expected, the DEA technical model has the lowest correlations with the three 
economic efficiency models. 

Table 2 shows the distribution of school district expenditures and effectiveness scores 
across the quartiles of the four efficiency models. The distributions show that the efficiency 
models are more sensitive to expenditure than they are to achievement. This is not surprising 
given the lack of a significant correlation between school district effect and expenditure. 

Figures 8 and 9 illustrate graphically these inter-relationships for the two DEA models. 

The figures have the same configuration as Figure 6; but the districts are indicated by symbols 
reflecting the DEA quartiles in which they fall: a circle (first quartile, most efficient), a diamond 
(second), a triangle (third), and an inverted triangle (fourth, least efficient). 
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Table 1 . Pearson Correlation Coefficient Matrix for the Four Efficiency Models 



Correlations 


Quadriform 


Ratio 


DEA Efficiency 


DEA Technical 


Quadriform 


1.000 


.737 


.535 


.275 






p=.000 


p=.000 


p=.005 


Ratio 


.737 


1.000 


.721 


.361 




p=.000 




p=.000 


p=.000 


DEA Efficiency 


.535 


.721 


1.000 


.433. 




p=.000 


p=.000 




p=.000 


DEA Technical 


.275 


.361 


.433 


1.000 




p=.005 


p=.000 


p=.000 





School Efficiency 

The school efficiency analysis is the most tentative of the analyses we did because of the limited 
amount of data available at the school level. Output measures are available from the state tests as 
for the district analyses; but, for the input measures, only the numbers of personnel in three 
categories (teachers, para-professionals, and other professionals) were known to us. This type of 
analysis will become much more valuable as the quality and scope of the data improve. We report 
our findings here to illustrate the kind of policy relevant information efficiency analyses at the 
school level may yield in the future. 

School efficiency was measured using data envelopment analysis and ratio analysis. In 
both cases, only technical efficiency was addressed since adequate expenditure data at the school 
level were not available, thereby preventing economic efficiency analyses. The large number of 
grade configurations in the schools prevented us from comparing all schools simultaneously. 
Instead, it was necessary to divide the schools into five categories based on the grades offered; 
Elementary Schools; Middle Schools; Junior/Senior High Schools; High Schools; and. 
Comprehensive Schools. The distribution of schools among the categories is reported in Table 3. 
Due to the small number of schools compared to the number of tests, no efficiency analyses were 
conducted for Comprehensive Schools. 

Ratio Analysis . In calculating the ratios, an average of the available school effect 
measures for each school was used as the output measure (numerator). Inputs were measured 
using the number of persons employed in each personnel category per student enrolled; e g., 
teachers per student). The results were weighted and combined to create the denominator of the 
ratio. The weights used to combine the personnel were 1.0 for teachers, 0.5 for para- 
professionals and 1.2 for other professionals. These weights are arbitrary; however, there is little 
evidence to either refute or support these, or other, weights. (See discussion below.) 
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Table 2. Distribution of School District Expenditure and Effectiveness for Four Efficiency Models 

by Quartile 



DEA 
Technical 
Quartile 1 
Quartile 2 
Quartile 3 
Quartile 4 


1_ 

10 

5 

6 
4 


Expenditures 
<- lowest to hiqhest-> 


Total 

25 

26 
25 
25 


£ 

8 

3 

9 

5 


Effectiveness 
<- hiqhest to lowest -> 


Total 

25 

26 
25 
25 


2. 

9 

9 

5 

3 


3. 

2 

8 

9 

6 


£ 

4 

4 

5 

12 


2. 

8 

7 

5 

6 


£ 

8 

7 

4 

6 


£ 

1 

9 

7 

8 


Total 


25 


26 


25 


25 


101 


25 


26 


25 


25 


101 






Expenditures 






Effectiveness 




DEA 




<- lowest to hiqhest-> 






<- hiqhest to lowest -> 




Economic 


1_ 


2. 


3. 




Total 


£ 


2. 


£ 


£ 


Total 


Quartile 1 


19 


4 


2 


0 


25 


8 


8 


5 


4 


25 


Quartile 2 


5 


13 


6 


2 


26 


6 


8 


5 


7 


26 


Quartile 3 


1 


7 


10 


7 


25 


8 


5 


8 


4 


25 


Quartile 4 


0 


2 


7 


16 


25 


3 


5 


7 


10 


25 


Total 


25 


26 


25 


25 


101 


25 


26 


25 


25 
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Expenditures 
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<- lowest to hiqhest-> 
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Ratio 


1_ 


2. 


2 _ 


£ 


Total 


£ 


2. 


£ 


£ 


Total 


Quartile 1 


16 


5 


3 


1 


25 


13 


9 


2 


1 


25 
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5 


8 


7 


6 


26 


9 


9 


6 


2 


26 
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2 


10 


9 


4 


25 


1 


6 


11 


7 


25 
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2 


3 


6 


14 


25 


2 


2 


6 


15 


25 
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25 


26 


25 


25 
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25 


26 


25 


25 
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25 
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26 
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25 
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25 


25 
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25 
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DEA Technical Efficiency 
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Figure 8. Location of Districts According to DEA Technical Efficiency Quartile in Bivariate 
Distribution of Average School Effect and Adjusted Expenditure per Educational Need Unit 
(both measures are represented in standard scores) 
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Figure 9. Location of Districts .According to DEA Economic Efficiency Quartile in Bivariate 
Distribution of Average School Effect and Adjusted Expenditure per Educational Need Unit 
(both measures are represented in standard scores) 
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30 School and School District Efficiency 

Table 3. Distribution of Schools by Category for Efficiency Analysis 

Elementary Schools. Schools reporting scores for the grade 3 or 4 tests and possibly other tests 
for the elementary grades 

• 10 schools reporting scores for grade 3 only 

• 34 schools reporting scores for grades 3 and 4 

• 92 schools reporting scores for grades 3 through 5 

• 46 schools reporting scores for grades 3 through 6 

• 40 schools reporting scores for grades 3 through 8 

• 3 schools reporting scores for grades 4 and 5 

• 4 schools reporting scores for grades 4 through 6 

• 1 school reporting scores for grades 4 through 8 

Middle Schools. Schools reporting test scores for grades 5, 6 or 8 

• 12 schools reporting scores for grades 5 through 8 

• 8 schools reporting scores for grades 5 and 6 

• 36 schools reporting scores for grades 6 and 8 

• 6 schools reporting scores for grade 8 only 

Junior/Senior High Schools. Junior high school grades as well high school in the same building 
Schools included in this group reported test scores for grades 5, 6 or 8 as well as the high school 
Regents exams. 

• 10 schools reporting scores for grades 5 or 6 through 8 plus high school 

• 36 schools reporting scores for grade 8 plus high school 

High Schools. This group included high schools only. 

• 60 schools reporting scores for Regents examinations only 

Comprehensive Schools. Thirteen schools provide classes from kindergarten through grade 12. 
Due to the small number of schools compared to the number of tests, no efficiency analyses were 
conducted for these schools. Twelve of these schools were in rural areas and one was in a small 
city. 
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School and School District Efficiency 3 1 

Table 4 provides the distribution of ratio-efficiency of schools by quartiles (1 being the 
most efficient and 4 the least efficient) by school classification and district type (Buffalo, small 
city, suburban/rural [non-city school districts in Chautauqua, Erie, Genesee, Niagara, and Orleans 
Counties], and rural [non-city school districts in Allegany, Cattaraugus and Wyoming Counties]), 
Both the number of schools included in each quartile and the percents of the total number of 
schools in the category are provided. For the elementary schools, most of the efficient schools are 
small city and suburban/rural schools, representing 39% and 30%, respectively, of the schools of 
that type included in the elementary group. In general, these schools are above average in school 
effect and well below average in the number of personnel working in the schools. The least 
efficient schools are primarily in Buffalo and suburban/rural area, representing 50% and 22%, 
respectively, of the schools of those types included in the elementary category. These schools are 
well below average in school effect and above average in terms of the number of personnel 
employed. Schools in the second and third quartiles are close to average in school effect. The 
schools in the third quartile tend to have more personnel in each category than those in the second 
quartile (which has more than those in the first quartile). Table 5 organizes the statistics in 
reverse order, that is, by district type and school classification. 

For the other three school categories. Tables 4 and 5 indicate that the suburban/rural and 
rural schools have more schools in the more efficient quartiles (1 and 2) than in the less efficient 
quartiles. In particular, for middle schools and high schools, approximately two-thirds of the 
schools in these groups are in quartiles 1 and 2. On the other hand the urban schools (in Buffalo 
and the small cities) fare considerably worse with a majority of their schools classified as 
inefficient (quartiles 3 or 4). 

For the middle schools, the patterns of school effects and numbers of personnel were as 
expected and were similar to the elementary schools. However, for junior/senior high schools, the 
numbers of personnel increased steadily from the first to the fourth efficiency quartile, as 
expected, but the school effects were approximately the same (and below average) for the second, 
third and fourth quartiles. For high schools, the patterns were as expected (decreasing school 
effects and increasing personnel) as one moved from the first to the fourth quartile. 

Data Envelopment Analysis . Due to the different combinations of test scores reported 
for the elementary, middle and junior/senior high school categories, an average school effect was 
determined in relation to all available school effect scoresfor each school. It should be noted that 
for the junior/senior high school group, two DEA analyses were conducted, one using an 
elementary school effect score and a high school effect score and the other using an average 
school effect score based on all available school effects scores. The correlation between the two 
was quite high (r = .88); therefore, in the interest of consistency with the other categories 
reporting a variety of different test scores, only the results of the analysis based on a single school 
effect score are reported. The DEA evaluation for the high school group is based on the three 
separate school effect scores. In all cases school inputs are measured by the number of personnel 
per student enrolled for all three categories of personnel. 
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32 School and School District Efficiency 

Table 4. Representation in Ratio Efficiency Quartiles 
by 1) School Category and 2) District Type 



Elementary Schools 



District Tvpe 




Number of schools 






Percent 








01 01 


01 


01 


Total 


01 


01 


01 


01 


Total 


Buffalo 


3 


6 


13 


22 


44 


6.8 


13.6 


29.5 


50.0 


100 


Small city 


19 


17 


5 


8 


49 


38.8 


34.7 


10.2 


16.3 


100 


Suburban/Rural 


35 


26 


30 


25 


116 


30.2 


22.4 


25.9 


21.6 


100 


Rural 


0 


9 


9 


3 


21 


0.0 


42.9 


42.9 


14.3 


100 


Total 


57 


58 


57 


58 


230 












Middle schools 






















District Tvpe 




Number of schools 








Percent 








01 


01 


01 


01 


Total 


01 


01 


01 


01 


Total 


Buffalo 






1 


1 


2 


0.0 


0.0 


50.0 


50.0 


100 


Small city 




2 


6 


8 


16 


0.0 


12.5 


37.5 


50.0 


100 


Suburban/Rural 


14 


13 


7 


7 


41 


34.1 


31.7 


17.1 


17.1 


100 


Rural 


1 


1 


1 






'y 'y 'y 


33.3 


33.3 


0.0 


100 


Total 


15 


16 


15 


16 


62 












Junior/ Senior High Schools 


















District Tvpe 




Number of schools 








Percent 








01 


01 


01 


01 


Total 


01 


01 


01 


01 


Total 


Buffalo 


1 




1 


2 


4 


25.0 


0.0 


25.0 


50.0 


100 


Small city 








1 


1 


0.0 


0.0 


0.0 


100.0 


100 


Suburban/Rural 


7 


8 


6 


7 


28 


25.0 


28.6 


21.4 


25.0 


100 


Rural 


3 


4 


4 


2 


13 


23.1 


30.8 


30.8 


15.4 


100 


Total 


1 1 


12 


11 


12 


46 












High Schools 






















District Tvoe 




Number of schools 








Percent 








01 


01 


01 


01 


Total 


01 


01 


01 


01 


Total 


Buffalo 


1 




3 


1 


11 


9.1 


0.0 


27.3 


63.6 


100 


Small city 


1 


3 


3 


3 


10 


10.0 


30.0 


30.0 


30.0 


100 


Suburban/Rural 


13 


1 1 


8 


4 


36 


36.1 


30.6 


22.2 


11.1 


100 


Rural 




1 


1 


1 


3 


0.0 


33.3 


33.3 


3 3'. 3 


100 


Total 


15 


15 


15 


15 


60 
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School and School District Efficiency 33 

Table 5. Representation in Ratio Efficiency Quartiles 
by 1) District Type and 2) School Category 



Buffalo Number of schools Percent 





Q1 


01 


01 


Q4 


Total 01 


01 


01 


04 


Total 


Elementary 


3 


6 


13 


22 


44 


6.8 


13.6 


29.5 


50.0 


100 


Middle School 






1 


1 


2 


0.0 


0.0 


50.0 


50.0 


100 


Jr/Sr High School 


1 




1 


2 


4 


25.0 


0.0 


25.0 


50.0 


100 


High School 


1 




3 


7 


11 


9.1 


0.0 


27.3 


63.6 


100 


Total 


5 


6 


18 


32 


61 


8.2 


9.8 


29.5 


52.5 


100 


Small City 




Number of schools 








Percent 








Q1 


01 


01 


04 


Total 01 


01 


01 


04 


Total 


Elementary 


19 


17 


5 


8 


49 


38.8 


34.7 


10.2 


16.3 


100 


Middle School 


0 


2 


6 


8 


16 


0.0 


12.5 


37.5 


50.0 


100 


Jr/Sr High School 


0 


0 


0 


1 


1 


0.0 


0.0 


0.0 


100.0 


100 


High School 


1 


3 


3 


3 


10 


10.0 


30.0 


30.0 


30.0 


100 


Total 


20 


22 


14 


20 


76 


26.3 


28.9 


18.4 


26.3 


100 


Suburban/Rural 




Number of schools 








Percent 








01 


01 


01 


04 


Total 01 


01 


01 


04 


Total 


Elementary 


35 


26 


30 


25 


116 


30.2 


22.4 


25.9 


21.6 


100 


Middle School 


14 


13 


7 


7 


41 


34.1 


31.7 


17.1 


17.1 


100 


Jr/Sr High School 


7 


8 


6 


7 


28 


25.0 


28.6 


21.4 


25.0 


100 


High School 


13 


11 


8 


4 


36 


36.1 


30.6 


22.2 


11.1 


100 


Total 


69 


58 


51 


43 


221 


31.2 


26.2 


23.1 


19.5 


100 


Rural 




Number of schools 








Percent 








01 


01 


01 


04 


Total 01 


01 


01 


04 


Total 


Elementary 


0 


9 


9 


3 


21 


0.0 


42.9 


42.9 


14.3 


100 


Middle School 


1 


1 


1 


0 


3 


33.3 


33.3 


33.3 


0.0 


100 


Jr/Sr High School 


3 


4 


4 


2 


13 


23.1 


30.8 


30.8 


15.4 


100 


High School 


0 


1 


1 


1 


j 


0.0 


^ -> 


33.3 


33.3 


100 


Total 


4 


15 


15 


6 


40 


10.0 


37.5 


37.5 


15.0 


100 
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34 School and School District Efficiency 

Tables 6 and 7 report the results of the DEA efficiency evaluations. For the elementary 
schools, (Table 6) most of the efficient schools are small city and suburban/rural schools, 
representing 49% and 23%, respectively, of the schools of that type included in the elementary 
category. In general, these schools are above average in school effect and well below average in 
the number of personnel working in the schools. The least efficient schools are located primarily 
in Buffalo and the suburban rural area, representing 55% and 21%, respectively, of the schools of 
that type included in the elementary group. 

It should be noted that the representation of schools in the DEA efficiency quartiles is very 
similar to that of the ratio efficiency quartiles. Furthermore, examination of the pattern of relative 
school effects and the relative numbers of personnel is also very similar. A closer examination of 
the consistency of the two models in placing schools in efficiency quartiles indicates that 64% of 
schools are placed in the same quartile, and 95% are placed within 1 quartile under the two 
methods. This degree of consistency is noteworthy, but additional research would have to be 
undertaken to determine whether this would usually be the case for this type of analysis. 

Discussion . Thirty six elementary schools, thirteen middle schools, five junior/senior high 
schools and ten high schools were classified as efficient by both models. Of these schools, three 
were located in Buffalo, seventeen in small cities, forty in the suburban/rural area, and four in the 
rural area. These represent 5%, 22%, 18% and 10%, respectively, of the total schools in each of 
these categories. Furthermore, forty-five elementary schools, fifteen middle schools, ten 
junior/senior high schools, and ten high schools were classified as inefficient by both models. Of 
these, twenty-eight were located in Buffalo, sixteen in small cities, thirty-one in the suburban/rural 
area, and five in the rural area, representing 46%, 21%, 14% and 13% of the schools in each of 
these categories. 

We have a number of concerns with respect to these analyses at the school level. First, as 
mentioned earlier, the combination of personnel was arbitrary. However, a limited sensitivity 
analysis indicated that the rankings of schools based on the efficiency ratios change only slightly 
for changes in the weights for the para-professionals and the professionals. Nevertheless, the 
choice of appropriate weights is an issue that will require further attention. A more serious 
problem is that we do not have a mechanism for weighting the numbers of personnel for pupils 
with various disabilities in the schools. The number of additional personnel required for disabled 
pupils may not be adequately incorporated into our analysis. This is a significant limitation and 
may explain the relatively poor performance of urban schools with respect to both efficiency 
measures— and the relatively strong performance of the suburban districts. 

Policy Analysis Tools 

The development of indices in the previous sections of this report have provided information that 
can be of use to policy makers (e g., school boards, superintendents, and principals) in making 
global assessments of the effectiveness of their school operations in terms of student achievement 
adjusted and unadjusted for environmental factors. In this section, we provide a series of charts 
designed to assist in making comparisons among schools and school districts, that is. 
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Table 6. Representation in DEA Efficiency Quartiles by 1) School Category and 2) District Type 







Number of schools 








Percent 






District Tvpe 


Qi 


Q2 


01 


Q4 


Total 


Qi 


Q2 


01 


Q4 


Total 


Buffalo 


2 


6 


12 


24 


44 


4.5 


13.6 


27.3 


54.5 


100 


5mall city 


24 


9 


11 


5 


49 


49.0 


18.4 


22.4 


10.2 


100 


Suburban/Rural 


27 


38 


27 


24 


116 


23.3 


32.8 


23.3 


20.7 


100 


t^ural 


4 


5 


7 


5 


21 


19.0 


23.8 


33.3 


23.8 


100 


Total 


57 


58 


57 


58 


230 












Vliddle Schools 


























Number of schools 








Percent 






District Tvpe 


Qi 


Q2 


Q3 


Q4 


Total 


Qi 


Q2 


Q3 


Q4 


Total 


Buffalo 


0 


0 


2 


0 


2 


0.0 


0.0 


100.0 


0.0 


100 


5mall cit\' 


0 


2 


6 


8 


16 


0.0 


12.5 


37.5 


50.0 


100 


5uburban/Rural 


14 


13 


7 


7 


41 


34.1 


31.7 


17.1 


17.1 


100 


Rural 


1 


1 


1 


0 


3 


33.3 


33.3 


33.3 


0.0 


100 


Total 


15 


16 


16 


15 


62 












Junior/Senior High Schools 


























Number of schools 








' Percent 






District Tvpe 


Qi 


QI 


Q3 


Q4 


Total 


Qi 


Q2 


QI 


Q4 


Total 


Buffalo 


2 


0 


1 


1 


4 


50.0 


0.0 


25.0 


25.0 


100 


Small city 


0 


0 


0 


1 


1 


0.0 


0.0 


0.0 


100.0 


100 


Suburban/Rural 


6 


9 


6 


7 


28 


21.4 


32.1 


21.4 


25.0 


100 


Rural 


3 


3 


4 


3 


13 


23.1 


23.1 


30.8 


23.1 


100 


Total 


11 


12 


11 


12 


46 












Fligh School 


























Number of schools • 








Percent 






District Tvpe 


Qi 


QI 


Q3 


Q4 


Total 


Qi 


Q2 


QI 


Q4 


Total 


Buffalo 


3 


0 


2 


6 


11 


27.3 


0.0 


18.2 


54.5 


100 


Small cit\' 


1 


0 


5 


4 


10 


10.0 


0.0 


50.0 


40.0 


100 


Suburban/Rural 


11 


13 


8 


4 


36 


30.6 


36.1 


22.2 


11.1 


100 


Rural 


0 


2 


0 


1 


3 


0.0 


66.7 


0.0 


33.3 


100 


Total 


15 


15 


15 


15 


60 












\11 Categories 


























Number of schools 












Percent 


District T\pe 


Qi 


Q2 


Q3 


Q4 


Total 


Qi 


Q2 


01 


Q4 


Total 


Buffalo 


1 


6 


17 


31 


61 


11.5 


9.8 


27.9 


50.8 


100 


Small city 


25 


11 


22 


18 


76 


32.9 


14.5 


28.9 


23.7 


100 


Suburban/Rural 


58 


73 


48 


42 


221 


26.2 


33.0 


21.7 


19.0 


100 


Rural 


8 


11 


12 


9 


40 


20.0 


27.5 


30.0 


22.5 


100 


Total 


98 


101 


99 


100 


398 
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36 School and School District Efficiency 



Table 7. Representation in DEA Efficiency Quantiles 
by 1) District Type and 2) School Category 



Buflalo 


Number of schools 
^ Q2 Q3 


Q4 


Total 


01 


01 


Percent 

01 


Q4 


Elementary 


2 


6 


12 


24 


44 


4.5 


13.6 


27.3 


54.5 


Middle School 


0 


0 


2 


0 


2 


0.0 


0.0 


100.0 


0.0 


Jr/Sr High School 


2 


0 


1 


1 


4 


50.0 


0.0 


25.0 


25.0 


High School 


3 


0 


2 


6 


11 


27.3 


0.0 


18.2 


54.5 





7 


6 


17 


31 


61 


11.5 


9.8 


27.9 


50.8 


Small City 


01 


Number of schools 
Q2 Q3 Q4 


Total 


01 


01 


Percent 

01 


Q4 


Elementary 


24 


9 


11 


5 


49 


49.0 


18.4 


22.4 


10.2 


Middle School 


0 


2 


6 


8 


16 


0.0 


12.5 


37.5 


50.0 


Jr/Sr High School 


0 


0 


0 


1 


1 


0.0 


0.0 


0.0 


100.0 


High School 


1 


0 


5 


4 


10 


10.0 


0.0 


50.0 


40.0 




25 


11 


22 


18 


76 


32.9 


14.5 


28.9 


23.7 


Suburban/Rural 


01 


Number of schools 
Q2 Q3 Q4 


Total 


01 


01 


Percent 

01 


Q4 


Elementary 


27 


38 


27 


24 


116 


23.3 


32.8 


23.3 


20.7 


Middle School 


14 


13 


7 


7 


41 


34.1 


31.7 


17.1 


17.1 


Jr/Sr High School 


6 


9 


6 


7 


28 


21.4 


32.1 


21.4 


25.0 


High School 


11 


13 


8 


4 


36 


30.6 


36.1 


22.2 


11.1 




58 


73 


48 


42 


221 


26.2 


33.0 


21.7 


19.0 


Rural 


01 


Number of schools 
Q2 Q3 Q4 


Total 


01 


01 


Percent 

01 


Q4 


Elementary 


4 


5 


7 


5 


21 


19.0 


23.8 


33.3 


23.8 


Middle School 


1 


1 


1 


0 


3 


33.3 


33.3 


33.3 


0.0 


Jr/Sr High School 


3 


3 


4 


3 


13 


23.1 


23.1 


30.8 


23.1 


High School 


0 


2 


0 


1 


3 


0.0 


66.7 


0.0 


33.3 




8 


11 


12 


9 


40 


20.0 


27.5 


30.0 


22.5 




Total 

100 

100 

100 

100 

100 



Total 

100 

100 

100 

100 

100 



Total 

100 

100 

100 

100 

100 



Total 

100 

100 

100 

100 

100 
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School and School District Efficiency 37 

benchmarking. The charts will show policy makers where their resource allocation and 
performance profiles differ markedly from the norm for the western New York region. Differing 
from the norm is neither good nor bad in and of itself Differing significantly from the norm may 
be the key to success for highly efficient, high achieving districts. On the other hand, differing 
significantly from the norm for low achieving, low efficiency school districts may prove to be the 
key to failure. It isn’t the deviation from the norm, but rather the rationale for the deviation, that 
is critical. If the deviation is the result of deliberate district policy, the deviation is probably 
justified. More often than not, however, the deviation is the result of “drift” as a result, for 
example, of a changing profile of students and/or the teaching staff, response to highly effective 
advocates for a particular program, or labor agreements or contracts with vendors that do not 
work in the district’s best interest. Evaluating district policies that produce deviations from the 
norm is a good place to begin the long Journey to improving district performance and 
stewardship. 

Bernhardt (1998) notes that the power of charts and graphs comes from their ability to 
convey data directly to the viewer. Viewers use their spacial intelligence to retrieve data from a 
graph, complementing their language-based intelligence. Graphics display multiple measures, 
encouraging the eye to compare different pieces of data at several levels of detail. Graphs 
communicate information about relationships and the are especially effective in deepening or 
inducing understanding upon intensive inspection. They set the stage for discussion and analysis. 
“For most people, the communication process becomes more direct and immediate through 
graphic displays. Data become more credible and more convincing when the audience has direct 
interaction with it. Graphs allow us to move easily from the analytical to the descriptive and vice 
versa.” (P. 181). 

Analyzing School and School District Effectiveness 

The standard score chart for assessing districts’ relative performance with respect to 
aggregate student achievement and school effect was presented as Figure 2 (p. 9). The chart 
generated for assessing schools’ relative performance was presented as Figure 3 (p. 9)'*. 
Information usefial in interpreting profiles drawn on these charts was presented in the discussions 
on pages 7-14. 

Fifteen additional charts have been developed for analyzing relative performance at the 
school level. These were constructed the same way as the student achievement and school effect 
standard score charts were constructed, but they take on a different appearance. For these, only 
the standard deviation scale at the left appears the same. The bar for each achievement statistic 
shows the mean in raw scores (for example, actual average scores or percentages) as is the case 



■* Aggregated school achievement and school effect data for all schools and districts in the 
study can be downloaded as Lotus or Excel spreadsheets from 

h t to :/-'V'w^'. buffalo . edu/ gse/fas/ swansbn/649/ . Once you reach this location, scroll down to item 
#5 and follow the instructions. 
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38 School and School District Efficiency 

for the amounts at one standard deviation intervals above and below the mean. These charts are: 

I. Descriptive Information (1993-1996) 

II. Performance Related Measures (1992-1995) 

III. Grade 3 Reading & Mathematics Achievement (1993-1996) 

IV. Grade 4 Science Achievement (1995-1996) 

V. Grade 5 Writing Achievement (1993-1996) 

VI. Grade 6 Reading, Mathematics & Social Studies Achievement (1993-1996) 

VII. Grade 8 Social Studies Achievement (1995-1996) 

VIII. English Regents Achievement (1993-1996) 

IX. U. S. History & Government Regents Achievement (1993-1996) 

X Global Studies Regents Achievement (1993-1996) 

XI Languages Other Than English Regents Achievement (1993-1996) 

XII Mathematics I & III Regents Achievement (1993-1996) 

XIII Biology Regents Achievement (1993-1996) 

XIV Chemistry Regents Achievement ( 1 993-1 996) 

XV Introduction to Occupations Regents Achievement (1993-1996) 

Data to be used for plotting a school’s profile may be obtained from the school’s 1996 New 
School York State School Report Carcf. Profiles may be plotted for the school’s average 
achievement, the district’s average achievement, and the average achievement of schools 
statewide with similar demographic characteristics. The school’s profile may also be compared 
with reference schools of one’s own preference. For illustrative purposes, profiles are shown in 
subsequent figures for Drake Elementary School, Lowry Middle School, and North Tonawanda 

High School in the North Tonawanda School District. School average achievement ( ) 

in these illustrations are compared with district averages ( ) and average performance of 

similar schools( ). 



Drake Elementary School . Achievement Chart I, Figure 10, shows descriptive 
information about the percentage of pupils receiving free lunch (a measure of socio-economic 
status) and percent of pupils with limited English proficiency. In terms of “Percent Pupils 
Receiving Free Lunch,” Drake is nearly one standard deviation below the mean; in other words, it 
is relatively affluent for the region and slightly more affluent than other schools in the district. Its 



^1996 New York State School Report Card data for all Western New York schools and 
districts can be accessed via the internet at <http://rin. buffalo. edu/default.html>^ Upon reaching 
the site, follow the steps listed below: 

1. click on the hyperlink; 

2. click on the county in which the school is located; 

3. click on "Education;" 

4. click on "Public Schools;" 

5 click on the name of the school of interest; 

6. click on "New York State 1996 School Report Card" and find the data that you want to record. 
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Figure 10. Achievement Chart I; Descriptive Information (1993-1996), for Drake Elementary 

School and the North Tonawanda School District 
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Figure 11. Achievement Chart ELA: Performance Related Measures (1993-1996), 

for Drake Elementary School 
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Figure 12. Achievement Chart HB; Descriptive Information (1993-1996), 

for Drake Elementary School 
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40 School and School District Efficiency 

“Percent of Pupils with Limited English Proficiency” is zero, low for the region and for the 
district. 



Its “Attendance Rate” is generally high and more than one standard deviation above the 
regional mean for 1994-1995 as shown in Achievement Chart IIA (Figure 1 1). The “Suspension 
Rate” for Drake is zero as shown in Achievement Chart IIB (Figure 12). “Dropout Rate” and 
“Regents Diploma Rate” do not pertain to elementary schools. 

Chart III (Figure 13) shows Drake’s third grade achievement in mathematics and reading. 
Its performance tends to fall in the middle range. Chart IV (Figure 14) reports fourth grade 
science achievement and Chart V (Figure 14) reports on writing achievement in grade five; again, 
its performance tends to be in the middle range. 

Lowry Middle School . Achievement Charts I and II are repeated for Lowry Middle 
School (Figures 15, 16 and 17). Lowry’s student population is less affluent than Drake’s; its 
“Percent Pupils Receiving Free Lunch” is about average for the region and above the district 
percentage. For 1992-1993 and 1993-1994, its “Attendance Rate” was in the average range for 
the region; but, in 1994-1995, it showed a dramatic improvement. The school’s “Suspension 
Rate” falls into the middle range. 

Achievement Chart VIA (Figure 18) reports on reading and mathematics achievement for 
grade six. Its profiles fall into the middle range as do those for the district and similar schools. 
Figure 19 shows the profiles for Achievement Charts VLB & VII, social studies achievement for 
grades six and eight (1995-1996). The school’s performance falls into the middle range for grade 
six as do the reference groups. At grade eight, the school’s performance is consistently more than 
one standard deviation above the mean while the reference groups remain in the middle range. 
Figures 20 and 21 report performance on the Mathematics I Regents Examinations. For both 
percent mastery and percent passing, the school’s record is low for the district, but similar to that 
for similar schools. This may be the case because most students taking the Mathematics I Regents 
Examination are in ninth grade rather than eighth. 

North Tonawanda High School . Achievement Charts I and II are repeated for North 
Tonawanda High School (Figures 22, 23 and 24). The high school’s profiles on these charts tend 
to fall in the middle range for the distribution and are similar to those of similar schools. Figures 
25 through 33 (Achievement Charts VIII through XV) show the profiles of percent of students 
passing and percent achieving mastery on each Regents examination. The pupils of the school 
consistently achieve in the middle range of the distribution. 

Analvzine School District Fiscal Policy 

Charts designed to assist in assessing school district fiscal policy were constructed the 
same way as the student achievement and school effect standard score charts were constructed. 
The standard deviation scale appears at the left of each chart. The bar for each expenditure 
category shows the mean in raw scores (for example, actual dollars, percentages, etc.) as is the 
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Figure 13. Achievement Chart III: Grade 3 Reading and Mathematics Achievement 
(1993-1996), for Drake Elementary School, the District and Similar Schools 
This School ( -) This District ( ) Similar Schools ( ) 



Rending 
I ‘/4.9S 



Reading 
V* Above 

I 1“ 2“ Mattrnj 

Int, u.fl Int. IvArl 

1‘JS-yfc S-Vfc j 



Maiitrniaiics 

*.V Ah«n> loH 

I V4- JS -JS ■■■■'. 1 



■ 10 



-TIU 

• 71 .* 



5 




It 



Figure 14. .Achievement Charts IV & V: Grade 4 Science Achievement (1995-1996) 
and Grade 5 Writing .Achievement (1993-1996), for Drake Elementary School, 

the District and Similar Schools 

This School ( ) This District ( ) Similar Schools ( ) 
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Figure 15. Achievement Chart I; Descriptive Information (1993-1996), for Lowry Middle 
School and the North Tonawanda School District 
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Figure 16. Achievement Chart HA; Performance Related Measures (1993-1996), 

for Lowry Middle School 
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Figure 17. Achievement Chart HB: Descriptive Information (1993-1996), 

for Lowry Middle School 
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Figure 18. Achievement Chart VIA; Grade 6 Reading and Mathematics Achievement 
(1993-1996), for Lowry Middle School, the District and Similar School 
This School ( ) This Disuict ( ) Similar Schools ( ) 
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Figure 19. Achievement Charts VTB & VH; Grades 6 and 8 Social Studies Achievement 
(1995-1996), for Lowry Middle School, the District and Similar Schools 
This School ( ) This District ( ) Similar Schools ( ) 
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Figure 20. Achievement Chart XIIA; Percent Mastery of Mathematics I and III Regents 
Achievement (1993-1996), for Lowry Middle School, the District and Similar Schools 
This School ( This District ( ) Similar Schools ( ) 
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Figure 21. Achievement Chart XllB Percent Passing Mathematics I and EQ Regents 
Achievement (1993-1996), for Lowry Middle School, the District and Similar Schools 
This School. ( ) This District ( ) Similar Schools ( 
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Figure 22. Achievement Chart I: Descriptive Information (1993-1996), for North Tonawanda 
High School and the North Tonawanda School District 
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Figure 23. Achievement Chart ELA: Performance Related Measures (1993-1996), 

for North Tonawanda Hiah School 
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Figure 24. Achievement Chart IIB: Descriptive Information (1993-1996), for 
North Tonawanda Hiah School and Similar Schools 
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Figure 25. Achievement Chart VIII: English Regents Achievement (1993-1996), 
for North Tonawanda High School and Similar Schpols 
This School ( ) Similar Schools ( ) 

•/• Mastery % Passing 
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Figure 26. Achievement Chart EX; U. S. History and Government Regents Achievement 
(1993-1996), for North Tonawanda Eiigh School and Similar Schools 
This School ( ) Similar Schools ( ) 
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Figure 27. Achievement Chart X: Global Studies Regents Achievement (1993-1996), 
for North Tonawanda High School and Similar Schools 
This School ( ) Similar Schools ( ) 
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Figure 29. Achievement Chart XIIA: Percent Mastery of Mathematics I and III Regents 
Achievement (1993-1996), for North Tonawanda High School and Similar Schools 
This School ( ) Similar Schools ( ) 
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Fi^re 30. Achievement Chart XllB: Percent Passing Mathematics I and EH Regents 
Achievement (1993-1996), for North Tonawanda High School and Similar Schools 
This School ( ) Similar Schools ( ■ 
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Figure 31. Achievement Chart XIII: Biology Regents Achievement (1993-1996), 
for North Tonawanda High School and Similar Schools 
This School ( ) Similar Schools ( ) 
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Figure 33. Achievement Chart XV; Introduction to Occupations Regents Achievement 
(1993-1996), for North Tonawanda High School and Similar Schools 
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case for the amounts at one standard deviation intervals above and below the mean. Expenditures 
shown in these charts have not been adjusted for variation in teacher costs within the region. 

Six standard score charts have been developed to evaluate district fiscal policy: 

I. District Ability and Effort, 1995-96 

II. Revenue Source by Level of Government, 1995-96 

III. Sources of Local Revenue, 1995-96 ($ per EDNEED) 

IV. Type of State .'Vd, 1995-96 ($ per EDNEED) 

V. Expenditure Overview, 1995-96 ($ per EDNEED) 

VI. Expenditures for Instruction, 1995-96 ($ per EDNEED) 

Plotting a school district’s information on the standard score charts designed for assessing 
fiscal policy provides a visual profile of how the district’s fiscal posture compares with other 
districts in the Western New York region. Financial data for all districts in the study except 
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Buffalo may be obtained from Financing Our Schools 1997, published by Erie #1 BOCES^. For 
additional comparisons, reference districts can be plotted on the same chart. For illustrative 
purposes, the profiles for the Buffalo, Frontier and Williamsville school districts are presented in 
Figures 34-39. Buffalo and Williamsville are the region’s two largest school districts and Frontier 
is the largest school district that consistently appears in the first quartile of efficiency on all 
efficiency models. 

District Ability and Effort . Figure 34 shows the profiles for the three districts in 
reference to tax raising ability and effort. “Combined Wealth Ratio” is a statistic used in several 
state aid formulas. It reflects the wealth of a district in terms of both household personal income 
and the market valuation of real property per pupil wealth unit (TWPU). Williamsville ranks very 
high on this statistic, being more than two standard deviations above the mean for the region. 
Buffalo and Frontier, on the other hand, are in the middle range (the grey area), Buffalo being 
slightly below the mean for the region and Frontier slightly above. Buffalo and Frontier also hold 
similar positions for “Income/TWPU” and “Full Value/T\WU.” Williamsville is above the middle 
range for both “Income/TWPU” and “Full Value/TWPU;” but, in relative terms, Williamsville is 
much wealthier in household personal income where it ranks among the top in the region than it is 
in real property valuation. The bar labeled “Tax Rate on Full Value” represents the effort that the 
taxpayers put forth in financially supporting their schools through the property tax. 

Williamsville’ s tax rate is in the middle range, slightly above average; Frontier’s tax rate is also in 
the middle range, but slightly below the mean. Buffalo’s tax rate for schools is one of the lowest 
in the region. 

Revenue Source by Level of Government . Figure 35 shows the sources of revenue 
coming from local taxpayers and state and federal governments in support of the General Fund 
which represents virtually all of the expenditures made for instructional purposes for most 
districts. “Total Revenue/EDNEED” represents the total amount of revenue collected by the 
district in support of the General Fund per pupil adjusted for student need of special services 
(EDNEED) as explained previously. All three districts fall in the middle range on this statistic 
with Williamsville slightly above the mean for the region and Buffalo and Frontier below the 
mean. Buffalo and Frontier are in the middle range in both dollar amount per EDNEED and 
percent coming from local, state and federal levels; although, Buffalo’s local revenue is nearly one 
standard deviation below the mean and its federal revenue is nearly one standard deviation above. 
Buffalo’s state aid is slightly above average for the region and Frontier’s is below. Williamsville’s 
profile is a good illustration of both the state and the federal governments’ “equalization” policies 
whereby priority is given to directing funds to districts with concentrations of poverty. Because 
of its wealth, even with only an average tax rate, Williamsville generates more local revenue per 
EDNEED than most other districts in the region; on the other hand, its state and federal support 
is among the lowest. 



^Alternatively, financial data for all districts in the study, including Buffalo, can be 
downloaded as Lotus or Excel spreadsheets from httn:/.'\vuw. buffalo.edu/gse/fas/swanson/649/. 
Click on “Readings and scroll down to item #6 and follow the instructions. 
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Figure 34. Profiles for Buffalo, Frontier and VVilliamsville School Districts for Finance Chart I 

District Ability and Effort, 1995-96 
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Figure 35. Profiles for Buffalo, Frontier and Williamsville School Districts for Finance Chart II 

Revenue Source by Level of Government, 1995-96 
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Figure 35 does not tell the whole story, however. While the General Fund contains nearly 
all of the instructional expenditures for most districts, some districts, especially those with high 
concentrations of pupils living in poverty like Buffalo, receive significant amounts of state and 
federal categorical aids that are accounted for in a Special Aid Fund. In the case of Buffalo, these 
Special Aid Fund revenues amounted to approximately $74,000,000 in addition to its General 
Fund Revenues of approximately $369,000,000 or 20%. Expenditures in 1995-96 from the 
Special Aid Fund amounted to approximately $78,000,000 in addition to its General Fund 
expenditures of approximately $364,000,000 or 21%. This has the effect of raising state aid as 
shown in Figure 8 from 64% to 69% and raising federal aid from 1% to 7.5%. The local share 
drops from 26% to 23%. 

Sources of Local Revenue . Figure 36 shows the sources of local revenues per EDNEED 
for the General Fund. The districts are most widely dispersed on the amount of revenue raised 
through the property tax which reflects the distribution of Full Value per TWPU reported in 
Figure 34. All three districts are above the middle range for “Non-Property Tax” which is derived 
primarily, if not exclusively from the portion of the Erie County sales tax dedicated to schools. 

All Erie County schools will be high on this statistic because Erie County is the only county in the 
region to dedicate a portion of its sales tax in support of schools. Buffalo is low on the “Use of 
Money and Property” which is a function of the fact that Buffalo is fiscally dependent on the city 
government and returns on investments go directly to the city and not to the district. All three 
districts are low on “Miscellaneous” items. Frontier’s revenue receipts are one standard deviation 
above the mean for “Charges for Services” and for “Sale of Property .” 

Types of State Aid . Figure 37 features the type of state aid received by districts. The 
high risk characteristics of Buffalo’s school population is reflected in the relatively high amounts 
of state aid it gets for pupils with “Extraordinary Needs” and pupils with “Disabilities” in “Public” 
facilities. Buffalo’s “Building” aid is among the lowest in the region. Buffalo’s other state aid 
sources reported on the chart fall in the middle range, although “Transportation” is nearly one 
standard deviation above the mean. Except for “Gifted and Talented” aid, Williamsville’s state 
assistance is below average for the region, and in three instances more than one standard deviation 
below the mean. Frontier’s state aid profile hovers near, but below, the region’s mean. Not 
included in the state aid reported in the figure is aid recorded in the Special Aid Fund, amounting 
to nearly $47,000,000 for Buffalo in 1995-96. 

General Fund Expenditures . Figure 38 features a breakdown of expenditures from the 
General Fund per EDNEED by major categories. In terms of “Total General Fund” 
expenditures, the three districts spend near the mean for the region. Williamsville’s expenditures 
for “Total Instruction” and “Plant & Maintenance” are more than one standard deviation above 
the mean. The rest of Williamsville’s expenditures are near the mean. Buffalo’s expenditures for 
“Plant and Maintenance,” Employee Benefits” and “Transportation” are more than one standard 
deviation above the mean. Buffalo’s other expenditures are in or near the middle range for the 
region. All of Frontier’s expenditures fall into the middle range. 

General Fund Expenditures for Instruction . Figure 39 enables a more detailed analysis 
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Figure 36. Profiles for Buffalo, Frontier and Williamsville School Districts for Finance Chart III: 
Sources of Local Revenue ($ per EDNEED), 1995-96 
Buffalo ( ) Frontier ( ) Williamsville ( ) 



Property , Non- Other 

Tax property Tax 

Tax Items 



Charges 

for 

Services 



Use of 
Money 
Prop. 



Sale Mis- lutcr- 

of cellnii* fund 

Property eons Transfer 



8618 



299 527 



305 44 



198 198 




Figure 37. Profiles for Buffalo, Frontier and Williamsville School Districts for Finance Chan IV: 

Types of State Aid ($ per EDNEED), 1995-96 
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Figure 38. Profiles for Buffalo, Frontier and Williamsville School Districts for Finance Chart V: 

Expenditure Overview, 1995-96 ($ perEDNEED) 
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Figure 39. Profiles for Buffalo, Frontier and Williamsville School Districts for Finance Chart VI: 
Expenditures for Instruction, 1995-96 ($ per EDNEED) 
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of the major expenditure component of the General Fund, “Instruction.” Frontier’s expenditures 
for Instruction categories hover near, but below, the mean for the region. Buffalo spends more 
than one standard deviation above the mean for “Administration & Improvement” and “Teaching 
Students with Disabilities” and more than one standard deviation below the mean for “Teaching 
Regular School” and “Pupil Services.” Williamsville’s expenditures for “Total Teaching” and 
“Teaching Regular School” are more than one standard deviation above the mean; its other 
expenditures are above average, but in the middle range for the region. 

Combined Expenditures from General Fund and Special Aid Fund . Expenditures 
made from the Special Aid Fund were not included in the finance charts because they are not 
available in the database used for the study. To illustrate the distortions that can be introduced 
into analysis of fiscal policy through the omission of expenditures made possible by the special 
aids, we obtained the Annual Financial Reports, New York State ST-3 Forms, for the two largest 
school districts in the region; Buffalo, which receives significant amounts of Special Fund Aid, 
and Williamsville, which receives relatively little as noted above. Figures 38 and 39 show Finance 
Charts V and VI, respectively, with the expenditure profiles for the two districts when 
expenditures from the Special Aid Fund are combined with those from the General Fund. For 
Williamsville, there is little change in the profiles; but, for Buffalo, the change is dramatic. In 
Figure 40, Finance Chart V, Buffalo’s expenditures for “Total Instruction” rise from slightly 
below average for the region to one and one-half standard deviations above the mean and “Total 
Employee Benefits” rise to more than two standard deviations above the mean. With respect to 
the subcategories of “Total Instruction,” reported in Figure 41, “Total Teaching” and “Teaching 
Regular School” join “Administration & Improvement” and “Teaching Students with Disabilities” 
at a level more than one standard deviation above the mean. Expenditures for “Total Pupil 
Services” and “Total Instructional Media” remain below average for the region. 

Future Research 

In many respects, the most difficult aspect of this research was the assembling of data 
bases from a number of sources. The work of Erie #1 BOCES in developing a data warehouse 
should make the assembling of databases almost routine, leaving more time for analysis and 
enabling annual reanalyses. Other regions and states should follow the Erie #1 example. 

We are relatively satisfied with the performance of the student achievement and school 
effect measures although we still need to look for refinements. For example, the relationship 
between socio-economic status and pupil achievement may be curvilinear. If this is the case, we 
may have over corrected, thus understating the school effect of high socio-economic schools and 
districts. We did test for curvilinearity in this analysis and found the linear assumption to be 
reasonable. Never-the-less, the precise nature of the relationship should continue to be studied. 

If these analyses were to be conducted annually, we would need to agree upon 
procedures and weights to allow for stability of measures from year to year. To establish weights 
for calculating school effect on a continuing basis, the regression equations for several years need 
to be studied. The averaging procedure used in developing the student achievement measures at 
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Figure 40. Profiles for Buffalo and Williarnsville School Districts for Finance Chart V: 
Expenditure Overview, 1995-96 ($ per EDNEED) 

(with General Fund and Special Aid Fund expenditures combined) 
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Figure 41. Profiles for Buffalo and Williamsville School Districts for Finance Chart VI: 
Expenditures for Instruction, 1995-96 (S per EDNEED) 

(with General Fund and Special Aid Fund expenditures combined) 
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the school level worked quite well and may be preferable to the factor analysis procedure used for 
district student achievement scores because the methodology is more easily understood by 
persons not having an extensive background in statistical methods. 

It would be interesting to use Hierarchal Linear Modeling (HLM) to determine the nested 
effects of varying levels of efficiency for schools of the same level for the same district. This 
might provide insight as to why Buffalo, for example, has such a skewed distribution of efficiency 
scores among its elementary schools. 

We are also convinced that financial data need to be adjusted for within region cost 
differences. In future analyses, these adjustments should be applied to the financial charts as well 
as to the efficiency analyses. A Cost of Education Index (CEI) is in the process of being 
developed by the National Center for Education Statistics that includes all costs, not just those for 
teaches (Chambers, 1998). When CEI indices are available at the district or county levels, they 
should be used instead of the Teacher Cost Index as was used in this analysis. 

The EDNEED measure worked well in accounting for the differences in level of 
educational services required by children with different characteristics. While EDNEED reflects 
current New York State policy, it should be monitored in future applications to assess continuing 
satisfactory performance because a number of other studies suggest that it may be too 
conservative. Also, the “TAPU for Expense” measure used in calculating some state aids should 
be evaluated as a substitute for EDNEED. “TAPU for Expense” was not available in the database 
used for the study; but, if the two measures prove to be comparable, one that is widely used and 
understood is preferable to a new measure. 

In preparing the finance charts, expenditures from the Special Aid Fund should be 
included in the reporting. Most federal aids are placed in this fund rather than the General Fund 
as are some state aids. Where these aids are large, a false picture of a district’s fiscal profiles is 
presented when the services purchased with them are not included among the district’s 
expenditures. All statistical analyses and charts heed to be redesigned as better information 
becomes available. An advisory committee of persons using the analyses in making school district 
policies should participate in the redesign process. 

The most innovative portion of the study is the efficiency analysis. The quadriform and 
ratio approaches are easily understood and can be applied immediately in analyzing school district 
fiscal policy. The DEA analyses have great potential for identifying fiscal slack in a school 
district; but it needs further development before it can be applied with confidence. 

Despite our lack of full confidence in our efficiency measures, a dozen districts (both rural 
and suburban) have been identified as being consistently more efficient in the use of resources 
than the other districts. A next step is to carefully study these districts to find out what they are 
doing that permits them to achieve above average school effect with only a modest application of 
resources. Findings could be helpful to other school districts in making their operations more 
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cost-effective. Enabling this type of research was the ultimate objective of the present research. 

Conclusions 

The information generally available for making school district comparisons has left much 
of the story untold. Some of the analyses that have been done and published by the mass media 
may serve a symbolic purpose, but they are of little strategic value in helping people to make 
beneficial changes in our educational systems. This study has added two dimensions, school 
district effect and school district efficiency, to the one normally considered, student achievement. 
In the process, we have highlighted the differences in challenges that school districts face. When 
more dimensions are considered, it is readily apparent that many school districts, not normally 
recognized for their effectiveness, are making important contributions to the intellectual 
development of their students. The contribution of other districts commonly recognized as 
outstanding are put into perspective. This does not, however, relieve schools and districts that 
have high school effects but whose students are still failing to meet established achievement 
standards of the responsibility for bringing those students up to standard; it just further illustrates 
the magnitude of the task. This study has shown that more spending on education is not the sole 
means to better results; money has to be spent wisely in order to achieve the results desired. 

This is where the concept of efficiency becomes very important. 

To assist in the difficult task of assessing school district fiscal policy, we have developed a 
series of charts to facilitate the process of benchmarking through making external comparisons. 
We have also developed a series of achievement charts that enable schools to visualize their actual 
contribution to student achievement and to make comparisons of student achievement with 
selected reference groups. 

Considerable sophistication has been developed in measuring the equity of resource 
allocation among and within school districts. As public concern over efficiency of school 
operations grows, more sophistication needs to be gained in defining and measuring that concept. 
We hope that this exercise has contributed to the general understanding of the forces contributing 
to student achievement, to analyzing efficiency of school operations, and to the need for further 
research on the topic. 

Further, if an assessment is made that these statistics and charts are useful to school policy 
makers and to the general public in better understanding the decisions that have to be made, and 
in ultimately making better decisions, then judgements need to be made as to whether this type of 
analysis should be continued on a regular basis and how. 



I 



60 School and School District Efficiency 



References 

Abt Associates. (1993), Prospects: The Congressionallv mandated study of educational growth 
and opportunity . Washington, DC: U, S. Department of Education. 

Adler, M, Fetch, A., & Tweedie, J. (1989). Parental choice and educational policy . Edinburgh, 
Scotland: Edinburgh University Press, 

Anderson, D. M. (1996). Stretching the dollar: Increasing efficiency in urban and rural schools. 
In L, O. Picus & J. L, Wattenbarger, Eds., Where does the money go? Resource 
allocation in elementary and secondary schools . Thousand Oaks, CA: Corwin Press. 

Banker, R.D., Chames, A. Cooper, W.W., Swarts, J, & Thomas, D. A, (1989). An introduction 
to data envelopment analysis with some of its models and their uses. Research in 
Governmental and Nonprofit Accounting . 5. 125-164, 

Banker, R. D., Conrad, F. & Strauss, R. P. (1986). A comparative application of data 

envelopment analysis and translog methods: An illustrative study of hospital production. 
Management Science . 32. 30-44. 

Benson, C. S. (1978). The economics of public education (3rd ed ). Boston: Houghton Mifflin. 

Berne, R. & Stiefel, L. (1984). The measurement of equity in school finance . Baltimore: Johns 
Hopkins University Press, 

Bernhardt, V. L. (1998). Data Analysis for comprehensive schoolwide improvement . 

Larchmont, NY : Eye on Education. 

Bigenwald, M, M. (1977). An extension of Thorstein Veblin’s “Theory of the Leisure Class” to 
contemporary consumption of educational services . Unpublished doctoral dissertation. 
State University of New York at Buffalo. 

Bowlin, W.F., Chames, A., Cooper, W.W, & Sherman, H.D, (1985), Data envelopment analysis 
and regression approaches to efficiency estimation and evaluation. Annuals of Operation 
Research. 2. 113-138. 

Bridge, R. G , Judd, C. M., & Moock, P, R. (1979). The determinants of educational outcomes. 
The impact of families, peers, teachers and schools . Cambridge, MA. Ballinger. 

Brookover, W , & Lezotte, L. (1979). Changes in school characteristics coincident with changes 
in student achievement . East Lansing, MI: State University, College of Urban 
Development. 

Brown, F. (1990). The language of politics, education and the disadvantaged. In S. L. Jacobson, 
& J. A. Conway, (Eds.), Educational leadership in an age of reform , pp, 83-100, 

Chaikind, S. Danielson, L. C., and Brauen, M. L. (1993). What do we know about the cost of 
special education: A selected review. Journal of Special Education . 26(40). 344-370. 

Chambers, J. (1996), Public school teacher cost differences across the United States: 

Introduction to a teacher cost index. In W. J. Fowler, Jr, (Ed.), Developments in school 
finance. 1995 . U. S. Office of Education, National Center for Education Statistics, pp.21- 
32. . 

Chambers, J, (1998), Geographic variations in the prices of public school inputs . Washington, 
DC, National Center for Education Statistics, Working Paper No, 98-04, 



School and School District Efficiency 61 



Chambers, J , & Fowler, W. (1995). Public school teacher cost differences across the United 
States: Analvsis/methodoloev report (NCES 95-758). Washington, DC: U. S. 

Department of Education, National Center for Education Statistics. 

Chames, A., Cooper, W.W., and Rhodes, E. (1978). Measuring the efficiency of decision making 
units. European Journal of Operational Research . 2, 429-444. 

Chames, A., Cooper, W.W., Divine, D., Ruefli, & Thomas, D. (1989). Comparisons of DEA and 
existing ratio and regression systems for effecting efficiency evaluations of regulated 
electric cooperatives in Texas, Research in Governmental and Nonprofit Accounting . 5, 
187-210. 

Coleman, J. S. (1966). Equality of educational opportunity . Washington, DC: U. S. 

Government Printing Office. 

Consortium on Renewing Education (CORE). (1998). 20/20 vision: A strategy for doubling 
America’s academic achievement by the year 2020 . Nashville, TN: Peabody Center for 
Education Policy, Vanderbilt University. 

Deming, W. E. (1986). Out of the crisis . Cambridge, MA. MIT Press. 

Dreeben, R., & Thomas, J. A. (1980). The analysis of educational productivity. Volume I: 

Issues in microanalvsis . Cambridge, MA: Ballinger. 

Duncombe, W., & Yinger, J. (1998). Analysis of two educational policy changes in New York 
State: Performance standards and property tax relief In J. H. Wyckoff (Ed ), Educational 
finance to support high learning standards: Final report . Albany, NY : The State Education 
Department, pp. 98-136. 

Edmonds, R. (1979). Effective schools for the urban poor. Educational Leadership . 37 . 15-24. 

Emihovich, C. (1994). Cultural continuities and discontinuities in education. In T. Husen & T. 

N. Postlethwaite (Eds.), The international encyclopedia of education (2nd ed., Vol. 3). 
Oxford, England: Pergamon, pp. 1227-1233. 

Erie No. One Board of Cooperative Educational Services. (Annually). Financing our schools . 
West Seneca, NY: Author. 

Ferguson, R. F. (1991). Paying for public education: New evidence on how and why money 
matters. Harvard Journal on Legislation . 28, 465-498. 

Frazier, A. (1997). A roadmap for quality transformation in education . Boca Raton, FL: St. 
Lucie Press. 

Ginsberg, R. & Berry, B. (1998). The capability for enhancing accountability. In R. J. S. 
Macpherson (Ed ), The politics of accountability: Educative and international 
perspectives . Thousand Oaks, CA: Corwin, pp.43-61. 

Grissmer, D. W., Kirby, S. N., Berends, M. and Williamson, S. (1994). Student achievement and 
the changing American family . Santa Monica, CA: RAND. 

Guthrie, J. W., Garms, W. I. & Pierce, L. C. (1988). School finance and education policy: 
Expanding educational efficiency, equity and choice . Englewood Cliffs, NJ: Prentice 
Hall. 

Hanushek, E. A. (1986). The economics of schooling: Production and efficiency in public 
schools. Journal of Economic Literature . 24 : 1141-1177. 

Hanushek, E. A. (1991). When school finance "reform" may not be good policy. Harvard 
Journal on Education . 28, 423-456. 




77 



62 School and School District Efficiency 



Hanushek, E. A. (1996). School resources and student performance. In G. Burtless (Ed ), Does 
money matter? The effect of school resources on student achievement and adult success . 
Washington, DC: Brookings Institution Press, pp. 43-73. 

Hedges, L. V., Laine, R. D., & Greenwald, R. (1994a). Does money matter? A meta-analysis of 
studies of the effects of differential school inputs on student outcomes. Educational 
Researcher . 23 (3). 5-14. 

Hendrie, C. ( 1999). A closer look at teachers . New York turns to the quality of teaching in the 
latest of its ongoing reform efforts. Education Week . 18 . 17, p. 163. 

Hickrod, G. A., et al. (1989). The biggest bang for the buck: An initial report on technical 
economic efficiency in Illinois K-12 schools with a comment on Rose v. The Council . 
Normal, IL: Center for the Study of Educational Finance (ERIC No. ED329013). 

Hitzges, R. A. (1988). Analyzing professional staff allocation decision process in selected public 
schools . Unpublished doctoral dissertation. State University of New York at Buffalo. 

Husen, T. (1972). Social background and educational career-research perspettives on equality of 
educational opportunity . Paris, France: Organization for Economic Cooperation and 
Development. 

Kennedy, M., Jung, R., & Orland, M. (1986). Poverty, achievement and the distribution of 
compensatory educational services . Washington, DC: U. S. Department of Education. 

Laine, R. D., Greenwald, R., & Hedges, L. V. (1996). Money does matter: A synthesis of a new 
universe of education production function studies. In L. O. Picus & J. L. Wattenbarger 
(Eds ), Where does the money go? Resource allocation in elementary and secondary 
schools . Thousand Oaks, CA: Corwin. 

Lankford, H., & Wyckoff, J. (1999). The allocation of resources to special education in New 
York State. In T. B. Parrish, J. G. Chambers, & C. M. Guarino, Funding special 
education . Thousand Oaks, CA: Corwin, pp. 147-175. 

Madaus, G. F., Airasian, P. W., & Kellaghan, T. (1980). School effectiveness: A reassessment 
of the evidence . New York: McGraw-Hill. 

Martinez, V., Thomas, K., & Kemerer, F.R. (1994). Who chooses and why: A look at five 
school choice plans. Phi Delta Kappan . 75 . 678-681. 

Mayeski, G. W., et al. (1972). A study of our nation's schools . Washington, DC: U. S. 
Government Printing Office. 

National Center for Educational Statistics. (1995). Disparities in public school district spending. 
1989-90 . Washington, DC: United States Department of Education. 

National Commission on Excellence in Education. (1983). A nation at risk . Washington, DC: 

U. S. Government Printing Office. 

New York State Education Department. (1995, 1997). New York, the state of learning. 

Statistical profiles of school districts . Albany, NY: The Department. 

Ogbu, J. (1978). Minority education and caste . New York: Academic Press. 

Parrish, T. B., & Hikido, C. (1998). Inequalities in public school district revenues . NCES 98- 
210. Washington, DC: U. S. Department of Education, National Center for Education 
Statistics. 

Pelavin, S., & Kane, M. (1990). Changing the odds: Factors increasing the access to college . 

New York: College Entrance Examination Board. 




78 



School and School District Efficiency 63 

RAND, Institute on Education and Training. (1994). Student performance and the changing 
American family. Policy Brief December. 1-3. 

Reschovsky, A. &. Imazeki, J. (1998). The development of school finance formulas to guarantee 
the provision of adequate education to low-income students. In W. J. Fowler, Jr. (Ed ), 
Developments in school finance 1997 . Washington, DC: Department of Education, 
National Center for Education Statistics, pp. 121-148. 

Rutter, M , Maughan, B. Mortimore, P , Ouston, J. (1979). Fifteen thousand hours. 
Secondary schools and their effects on children . London, England: Open Books. 

Sandowski, M. (1995). The numbers game yields simplistic answers on the link between 
spending and outcomes. The Harvard Education Letter . 1 1 (2), 1-4. 

Seiford, L. M. &. Thrall, R. M. (1990). Recent developments in DEA. Journal of Econometrics . 
46, 7-38. 

Sexton, T. R. (1986). The methodology of data envelopment analysis. In R. H. Silkman (Ed ), 
Measuring efficiency: An assessment of data envelopment analysis (pp. 7-31). San 
Francisco: Jossey-Bass Inc. 

Sexton, T. R., Silkman, R. H. &. Hogan, A. J. (1986). Data envelopment analysis: Critique and 
extensions. In R. H. Silkman (Ed ), Measuring efficiency: An assessment of data 
envelopment analysis fpp. 73-106V San Francisco: Jossey-Bass Inc. 

Sherman, H. D. (1986). Managing productivity of health care organizations. In R. H. Silkman 
(Ed), Measuring efficiency: An assessment of data envelopment analysis (pp. 3 1 -46). 

San Francisco: Jossey-Bass Inc. 

Shujaa, M. (1995). Cultural self meets cultural other in the African experience. Theory into 
Practice . 34 . 3, pp. 194-201. 

Schmoker, M. J. &. Wilson, R. B. (1993). Total quality education. Profiles of schools that 
demonstrate the power of Deming’s management principals . Bloomington, IN: 

Educational Foundation of Phi Delta Kappa. 

Summers, A. A., &. Wolfe, B. L. (1975). Equality of educational opportunity quantified: A 

production function approach . Philadelphia, PA: Federal Reserve Bank of Philadelphia, 
Department of Research. 

Swanson, A. D., &. King, R. A. (1997). School finance: Its economics and politics . New York: 
Longman. 

Taylor, C. (1998). Does money matter? An empirical study introducing resource costs and 
student needs to educational production function analysis. In W. J. Fowler, Jr. (Ed ), 
Developments in school finance. 1997. NCES 98-212. Washington, DC: U S. 
Department of Education, National Center for Education Statistics, pp. 75-97. 

Walberg, H. J. (1984). Improving the productivity of America's schools. Educational 
Leadership . ^ (May), 19-27. 

Wenglinsky, H. (1998). Finance equalization and within-school equity: The relationship between 
education spending and the social distribution of achievement. Educational Evaluation and 
Policy Analysis . 20, 269-283. 

Witte, J. F. (1993). The Milwaukee parental choice program. In E. Rasell &. R. Rothstein 

(Eds ), School choice: Examining the evidence, pp. 49-68. Washington, DC: Economic 
Policy Institute. 




79 



64 School and School District Efficiency 




SO 



School and School District Efficiency 65 



Glossary of Terms as Used in this Report 

Accountability. Public school officials being held responsible (answerable) for their fiscal 
performance and for the performance of the pupils in their charge. 

Achievement, Student. An aggregate measure of student academic performance made up of all 
state required tests given at grades 3, 4, 5, 6 and 8 and the per cent passing and per cent 
scoring at the mastery level or above on all Regents examinations. 

Benchmarking. Comparing current performance with a point of reference such as performance at 
a previous time for an organization or against the current performance of similar 
organizations; is a form of assessment that establishes external standards to which 
internal processes may be compared. 

Data Envelopment Analysis (DEA). Uses linear programming concepts to determine the 

efficiency of an organization in using resources in terms of outcomes received by adjusting 
data to prescribed theoretical requirements such as optimal production surfaces prior to 
undertaking various statistical tests. 

Educational Need Measure (EDNEED). An adjustment to pupil enrollment or attendance in a 

school or a district that takes into account the differences in costs of educational programs 
required to meet the instructional needs of children with varying characteristics, for 
example, special education and gifted students. 

Effect, District. A derived score that attempts to measure the influence of the educational 
experiences provided by a school district that are independent of peer and home 
influences. 

Effect, School. A derived score that attempts to measure the influence of the educational 
experiences provided by a school that are independent of peer and home influences. 

Effectiveness. The degree to which a school’s or district’s educational goals and objectives are 
realized. 

Efficiency. The ratio of outcomes (e g., school effect) to inputs (e.g., dollars per pupil in 
expenditure or professional personnel employed). 

Efficiency, Economic. The ratio of outcomes (e g., school effect) to inputs measured in terms of 
cost of resources consumed. 

Efficiency, Technical. The ratio of outcomes (e.g., school effect) to inputs measured in terms of 
the amount of resources consumed independent of their cost. 

Factor Analysis. A statistical procedure that identifies groups of variables that are highly 
correlated with one another. 

General Fund. The accounting fund in which virtually all operating expenditures are placed by 
most school districts. 

Multiple-regression. A statistical relationship between a single dependent variable (effect, e.g., 
student achievement) and a combination of two or more independent variables (cause, 
e.g., pupil-teacher ratio). 

Normal Curve. A symmetrical, bell shaped curve that describes the distributions of many random 
(chance) events. 

Normal Distribution. A distribution bounded by a normal curve wherein approximately two thirds 
of the cases fall between the two points of the curve’s inflexion. 

Outlier. A school or district that is characterized by being more than one standard deviation 
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above or below the mean of a group of schools or districts (a distribution) on a trait (e g., 
expenditure per pupil). 

Production Function, Educational. The causal relationship between inputs and outputs, e g., what 
goes on in schools and classrooms that directly affects the minds of students as measured 
by tests and other outcome measures. 

Production Possibility Frontier. The curve representing the most efficient use of a combination of 
resources. 

Quadriform Analysis. A procedure whereby schools or districts are grouped into quadrants 

according to the achievement of their pupils and the level of their resource commitment 
per pupil and the characteristics of the schools or districts within each quadrant are 
compared with those in other quadrants. 

Ratio Analysis. The ratio of a measure of school or district outcomes (e g., school effect) to a 
measure of school or district inputs (e.g., expenditure per pupil). 

Raw Score. An unadjusted measure. 

Regression Analysis. A statistical relationship between a single dependent variable (effect, e g., 
student achievement) and a single independent variable (cause, e g., pupil-teacher ratio). 

Residual Score. A measure (e.g., school effect) of the magnitude of a dependent variable (eg., 
student achievement) that is not related to (Or caused by) an independent variable (e g., 
socio-economic status). 

School District Ability. A measure of a school district’s wealth or ability to financially support its 
educational endeavors (e g., combined wealth ratio, market value of real property per 
pupil, household income per pupil). 

School District Effort. The proportion of its resources a school district uses in support of its 
educational enterprises (e g., tax rate on full value). 

Skewed. A distribution whose shape is not symmetrical. 

Socio-economic Status. The level of social esteem or affluence of an individual or family often 
measured by level of education or household income. 

Special Aids Fund. An accounting fund from which restricted revenues from state and federal 
governments are spent. 

Standard Deviation. A measure of variability whereby approximately two thirds of the cases of a 
distribution fall within one standard deviation of the mean (plus and minus) and 95 per 
cent of the cases will be within two standard deviations of the mean. 

Standard Score. A derived score that enables comparisons among several characteristics by 

providing the characteristics a common metric, their respective standard deviation, and a 
common mean, zero. 

Standards. A set of achievement expectation established by a state, school district, individual, etc. 

Teacher Cost Index. A ratio that adjusts for the variation in cost of teachers among school 
districts due to factors beyond their control such as cost of living, desirability of the 
community, etc. 
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Appendix A 

Standard Score Charts 
of 

Selected Financial Statistics 
for School Districts in the 
Western New York Region 



I. District Ability and Effort, 1995-96 

II. Revenue Source by Level of Government, 1995-96 

III. Sources of Local Revenue, 1995-96 ($ per EDNEED) 

IV. Type of State Aid, 1995-96 ($ per EDNEED) 

V. Expenditure Overview, 1995-96 ($ per EDNEED) 

VI. Expenditures for Instruction, 1995-96 ($ per EDNEED) 
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Finance Chart I. District Ability and EfTort, 1995-96 
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Finance Chart 111. Sources of Local Revenue ($/EDNEED), 1995-96 
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Finance Chart IV: State Aid by type, 1995-96 ($ per EDNEED) 
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Appendix B 

New York State 
School Report Card 
Standard Score Charts 

Based on the average scores of schools 
in the eight county region of Western New York 

I. Descriptive Information (1993-1996) 

II. Performance Related Measures (1992-1995) 

III. Grade 3 Reading & Mathematics Achievement (1993-1996) 

IV. Grade 4 Science Achievement (1995-1996) 

V. Grade 5 Writing Achievement (1993-1996) 

VI. Grade 6 Reading, Mathematics & Social Studies Achievement (1993-1996) 

VII. Grade 8 Social Studies Achievement (1995-1996) 

VIII. English Regents Achievement (1993-1996) 

IX. U. S. History & Government Regents Achievement (1993-1996) 

X Global Studies Regents Achievement (1993-1996) 

XI Languages Other Than English Regents Achievement ( 1 993- 1 996) 

XII Mathematics I & III Regents Achievement (1993-1996) 

XIII Biology Regents Achievement ( 1 993- 1 996) 

XIV Chemistry Regents Achievement (1993-1996) 

XV Introduction to Occupations Regents Achievement (1993-1996) 
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Achievement Chart XIIA. Percent Mastery Mathematics I and Ilf Regents Achievement (1993-1996) 
This School- This District Similar Schools 
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Acliieveineiit Chart XIIB. Percent Passing Mathematics I and III Regents Achievement (1993-1996) 
This School This District Similar Schools 
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Acilievciueiit Chart XIII. Biology Regents Achievement (1993-1996) 
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Achievement Chart XV. Introduction to Occupations Regents Achievement 
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Appendix C 



Standard Score Charts 
of 

Student Achievement 
and 

School Effects Statistics 
for Schools and School Districts in the 
Eight County Western New York Region 



Aggregated Student Achievement and School Effect for Grades 3, 4/5, 6, 8 
and Regents Examinations by School (in standard scores) 

Aggregated Student Achievement and School Effect for Grades 3, 4/5, 6 
and Regents Examinations by District (in standard scores) 
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Achieveinent Clinii XVI: Aggregated Student Acliieveineiit and School Efl'ect for Grades 3, 4/5, 6, 8 

and Regents Examinations for Schools (in standard scores) 
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